2024-02-13T14:02:40.332 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-13T14:02:40.332 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021 --owner scheduled_phlogistonjohn@teuthology --description orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} --name phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi --verbose -- /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/orig.config.yaml 2024-02-13T14:02:40.363 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T14:02:40.444 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021 branch: wip-phlogistonjohn-testing-2024-02-12-1654 description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} email: jmulligan@redhat.com first_in_suite: false job_id: '7558021' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-phlogistonjohn-testing-2024-02-12-1654 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-phlogistonjohn-testing-2024-02-12-1654 sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 owner: scheduled_phlogistonjohn@teuthology priority: 99 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1588 sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 sleep_before_teardown: 0 subset: 1/15 suite: orch:cephadm suite_branch: wip-phlogistonjohn-testing-2024-02-12-1654 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 targets: smithi055.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMYaR4eqleWPd3FOSNAoHSp4DmjLsuRLWUoySUWyHHtGDGN9bf9BFUKPQiNuoWOOrBvSz8N+z+znsbGLiv9D31Y= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLH6V8YJUcEN3NKh6RpRi6+fLcTt689WKZ7rQ0Ja4eYgguJG3wfyRt1PaYkNyj6NN3LOzAqTLTwXEojiA81Zi24= smithi170.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCNvjbm4hiy1kHXarPlsBzAe0C81UUSOvfK5nP94FfcfbqbWXTIWjmFj8PHGcDBPLKRDY3Drzhga22MMQ7ULxFM= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-13_11:41:43 tube: smithi user: phlogistonjohn verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-13T14:02:40.444 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa; will attempt to use it 2024-02-13T14:02:40.445 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_e1b79d5a4193a484176863ab96c0ca28c00e1662/qa/tasks 2024-02-13T14:02:40.445 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-13T14:02:40.446 INFO:teuthology.task.internal:Checking packages... 2024-02-13T14:02:40.469 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'e1b79d5a4193a484176863ab96c0ca28c00e1662' 2024-02-13T14:02:40.470 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-13T14:02:40.470 INFO:teuthology.packaging:ref: None 2024-02-13T14:02:40.470 INFO:teuthology.packaging:tag: None 2024-02-13T14:02:40.470 INFO:teuthology.packaging:branch: wip-phlogistonjohn-testing-2024-02-12-1654 2024-02-13T14:02:40.470 INFO:teuthology.packaging:sha1: e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:02:40.470 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-phlogistonjohn-testing-2024-02-12-1654 2024-02-13T14:02:40.683 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1423.ge1b79d5a 2024-02-13T14:02:40.684 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-13T14:02:40.695 INFO:teuthology.task.internal:no buildpackages task found 2024-02-13T14:02:40.695 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-13T14:02:40.708 INFO:teuthology.task.internal:Saving configuration 2024-02-13T14:02:40.719 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-13T14:02:40.730 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:02:40.756 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.734123', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMYaR4eqleWPd3FOSNAoHSp4DmjLsuRLWUoySUWyHHtGDGN9bf9BFUKPQiNuoWOOrBvSz8N+z+znsbGLiv9D31Y='} 2024-02-13T14:02:40.779 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.735323', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLH6V8YJUcEN3NKh6RpRi6+fLcTt689WKZ7rQ0Ja4eYgguJG3wfyRt1PaYkNyj6NN3LOzAqTLTwXEojiA81Zi24='} 2024-02-13T14:02:40.801 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.732770', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCNvjbm4hiy1kHXarPlsBzAe0C81UUSOvfK5nP94FfcfbqbWXTIWjmFj8PHGcDBPLKRDY3Drzhga22MMQ7ULxFM='} 2024-02-13T14:02:40.801 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-13T14:02:40.812 INFO:teuthology.task.internal:roles: ubuntu@smithi055.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-13T14:02:40.812 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-13T14:02:40.812 INFO:teuthology.task.internal:roles: ubuntu@smithi170.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-13T14:02:40.812 INFO:teuthology.run_tasks:Running task console_log... 2024-02-13T14:02:40.910 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fef83408f70>, signals=[15]) 2024-02-13T14:02:40.911 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-13T14:02:40.917 INFO:teuthology.task.internal:Opening connections... 2024-02-13T14:02:40.917 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi055.front.sepia.ceph.com 2024-02-13T14:02:40.918 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:41.024 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-02-13T14:02:41.025 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:41.111 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi170.front.sepia.ceph.com 2024-02-13T14:02:41.112 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:02:41.198 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-13T14:02:41.207 DEBUG:teuthology.orchestra.run.smithi055:> uname -m 2024-02-13T14:02:41.226 INFO:teuthology.orchestra.run.smithi055.stdout:x86_64 2024-02-13T14:02:41.226 DEBUG:teuthology.orchestra.run.smithi055:> cat /etc/os-release 2024-02-13T14:02:41.284 INFO:teuthology.orchestra.run.smithi055.stdout:NAME="CentOS Stream" 2024-02-13T14:02:41.284 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION="9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:ID="centos" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:ID_LIKE="rhel fedora" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:VERSION_ID="9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:PLATFORM_ID="platform:el9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:ANSI_COLOR="0;31" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:LOGO="fedora-logo-icon" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:HOME_URL="https://centos.org/" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-13T14:02:41.285 INFO:teuthology.orchestra.run.smithi055.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-13T14:02:41.286 INFO:teuthology.lock.ops:Updating smithi055.front.sepia.ceph.com on lock server 2024-02-13T14:02:41.306 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-02-13T14:02:41.323 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-02-13T14:02:41.323 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-02-13T14:02:41.380 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="CentOS Stream" 2024-02-13T14:02:41.380 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="9" 2024-02-13T14:02:41.380 INFO:teuthology.orchestra.run.smithi078.stdout:ID="centos" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="rhel fedora" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="9" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el9" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:LOGO="fedora-logo-icon" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://centos.org/" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-13T14:02:41.381 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-13T14:02:41.382 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-02-13T14:02:41.405 DEBUG:teuthology.orchestra.run.smithi170:> uname -m 2024-02-13T14:02:41.423 INFO:teuthology.orchestra.run.smithi170.stdout:x86_64 2024-02-13T14:02:41.424 DEBUG:teuthology.orchestra.run.smithi170:> cat /etc/os-release 2024-02-13T14:02:41.480 INFO:teuthology.orchestra.run.smithi170.stdout:NAME="CentOS Stream" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION="9" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:ID="centos" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:ID_LIKE="rhel fedora" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:VERSION_ID="9" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:PLATFORM_ID="platform:el9" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:ANSI_COLOR="0;31" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:LOGO="fedora-logo-icon" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:HOME_URL="https://centos.org/" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-13T14:02:41.481 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-13T14:02:41.482 INFO:teuthology.orchestra.run.smithi170.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-13T14:02:41.482 INFO:teuthology.lock.ops:Updating smithi170.front.sepia.ceph.com on lock server 2024-02-13T14:02:41.505 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-13T14:02:41.514 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-13T14:02:41.521 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-13T14:02:41.521 DEBUG:teuthology.orchestra.run.smithi055:> test '!' -e /home/ubuntu/cephtest 2024-02-13T14:02:41.523 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-02-13T14:02:41.525 DEBUG:teuthology.orchestra.run.smithi170:> test '!' -e /home/ubuntu/cephtest 2024-02-13T14:02:41.541 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-13T14:02:41.548 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-13T14:02:41.549 DEBUG:teuthology.orchestra.run.smithi055:> test -z $(ls -A /var/lib/ceph) 2024-02-13T14:02:41.581 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-02-13T14:02:41.583 DEBUG:teuthology.orchestra.run.smithi170:> test -z $(ls -A /var/lib/ceph) 2024-02-13T14:02:41.604 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-13T14:02:41.683 INFO:teuthology.run_tasks:Running task kernel... 2024-02-13T14:02:41.699 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-13T14:02:41.699 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-13T14:02:41.699 DEBUG:teuthology.orchestra.run.smithi055:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T14:02:41.700 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T14:02:41.700 DEBUG:teuthology.orchestra.run.smithi170:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-13T14:02:41.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:02:41.717 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-02-13T14:02:41.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:02:41.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:02:41.718 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-02-13T14:02:41.719 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-02-13T14:02:41.773 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-361.el9.x86_64 2024-02-13T14:02:41.773 INFO:teuthology.task.kernel:Running kernel on smithi078: 5.14.0-361.el9.x86_64 2024-02-13T14:02:41.774 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-02-13T14:02:41.775 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-361.el9.x86_64 2024-02-13T14:02:41.776 INFO:teuthology.task.kernel:Running kernel on smithi170: 5.14.0-361.el9.x86_64 2024-02-13T14:02:41.776 DEBUG:teuthology.orchestra.run.smithi170:> sudo yum install -y kernel 2024-02-13T14:02:41.777 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-361.el9.x86_64 2024-02-13T14:02:41.777 INFO:teuthology.task.kernel:Running kernel on smithi055: 5.14.0-361.el9.x86_64 2024-02-13T14:02:41.777 DEBUG:teuthology.orchestra.run.smithi055:> sudo yum install -y kernel 2024-02-13T14:02:44.053 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-13T14:02:44.072 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-13T14:02:44.174 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - AppStream 8.4 MB/s | 6.7 MB 00:00 2024-02-13T14:02:45.514 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS-9 - BaseOS 7.0 MB/s | 2.0 MB 00:00 2024-02-13T14:02:45.662 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS-9 - BaseOS 3.9 MB/s | 2.0 MB 00:00 2024-02-13T14:02:45.798 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-02-13T14:02:46.364 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-02-13T14:02:46.411 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-02-13T14:02:46.925 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-02-13T14:02:48.346 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-02-13T14:02:49.540 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - AppStream 9.2 MB/s | 19 MB 00:02 2024-02-13T14:02:50.548 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - AppStream 7.4 MB/s | 19 MB 00:02 2024-02-13T14:02:52.696 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 5.9 MB 00:00 2024-02-13T14:02:53.507 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - CRB 17 MB/s | 5.9 MB 00:00 2024-02-13T14:02:54.172 INFO:teuthology.orchestra.run.smithi078.stdout:CentOS Stream 9 - Extras packages 39 kB/s | 15 kB 00:00 2024-02-13T14:02:55.249 INFO:teuthology.orchestra.run.smithi055.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 15 kB 00:00 2024-02-13T14:02:55.916 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-02-13T14:02:57.583 INFO:teuthology.orchestra.run.smithi170.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 15 kB 00:00 2024-02-13T14:02:57.620 INFO:teuthology.orchestra.run.smithi055.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 20 MB 00:02 2024-02-13T14:03:00.226 INFO:teuthology.orchestra.run.smithi170.stdout:Extra Packages for Enterprise Linux 8.1 MB/s | 20 MB 00:02 2024-02-13T14:03:02.568 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 2.5 MB/s | 20 MB 00:08 2024-02-13T14:03:02.743 INFO:teuthology.orchestra.run.smithi055.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-02-13T14:03:05.126 INFO:teuthology.orchestra.run.smithi170.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-02-13T14:03:05.183 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-13T14:03:05.183 INFO:teuthology.orchestra.run.smithi055.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-13T14:03:05.282 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-02-13T14:03:05.290 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: Package Arch Version Repository Size 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:Installing dependencies: 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout:Install 4 Packages 2024-02-13T14:03:05.291 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:05.292 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 99 M 2024-02-13T14:03:05.292 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 122 M 2024-02-13T14:03:05.292 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-02-13T14:03:05.927 INFO:teuthology.orchestra.run.smithi055.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 9.5 MB/s | 6.0 MB 00:00 2024-02-13T14:03:06.620 INFO:teuthology.orchestra.run.smithi055.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 16 MB/s | 21 MB 00:01 2024-02-13T14:03:07.320 INFO:teuthology.orchestra.run.smithi055.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 24 MB/s | 33 MB 00:01 2024-02-13T14:03:07.441 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-13T14:03:07.441 INFO:teuthology.orchestra.run.smithi170.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-13T14:03:07.448 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-02-13T14:03:07.530 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout: Package Arch Version Repository Size 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-13T14:03:07.538 INFO:teuthology.orchestra.run.smithi170.stdout:Installing dependencies: 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout:Install 4 Packages 2024-02-13T14:03:07.539 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:07.540 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 99 M 2024-02-13T14:03:07.540 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 122 M 2024-02-13T14:03:07.540 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-02-13T14:03:07.686 INFO:teuthology.orchestra.run.smithi055.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-13T14:03:07.689 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:03:07.690 INFO:teuthology.orchestra.run.smithi055.stdout:Total 41 MB/s | 99 MB 00:02 2024-02-13T14:03:07.690 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-02-13T14:03:07.952 INFO:teuthology.orchestra.run.smithi170.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 15 MB/s | 6.0 MB 00:00 2024-02-13T14:03:08.127 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-02-13T14:03:08.128 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-02-13T14:03:08.610 INFO:teuthology.orchestra.run.smithi170.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-13T14:03:09.202 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-02-13T14:03:09.203 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-02-13T14:03:09.319 INFO:teuthology.orchestra.run.smithi170.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 24 MB/s | 33 MB 00:01 2024-02-13T14:03:09.763 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-13T14:03:09.763 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-13T14:03:09.786 INFO:teuthology.orchestra.run.smithi170.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 17 MB/s | 39 MB 00:02 2024-02-13T14:03:09.789 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:03:09.789 INFO:teuthology.orchestra.run.smithi170.stdout:Total 44 MB/s | 99 MB 00:02 2024-02-13T14:03:09.789 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-02-13T14:03:09.846 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-13T14:03:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout:Install 4 Packages 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 99 M 2024-02-13T14:03:09.854 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 122 M 2024-02-13T14:03:09.855 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-02-13T14:03:10.150 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-02-13T14:03:10.150 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-02-13T14:03:10.225 INFO:teuthology.orchestra.run.smithi078.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 16 MB/s | 6.0 MB 00:00 2024-02-13T14:03:10.900 INFO:teuthology.orchestra.run.smithi078.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 20 MB/s | 21 MB 00:01 2024-02-13T14:03:11.168 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-02-13T14:03:11.169 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-02-13T14:03:11.726 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-02-13T14:03:12.133 INFO:teuthology.orchestra.run.smithi078.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 18 MB/s | 33 MB 00:01 2024-02-13T14:03:12.422 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:12.562 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:12.583 INFO:teuthology.orchestra.run.smithi078.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 14 MB/s | 39 MB 00:02 2024-02-13T14:03:12.586 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:03:12.587 INFO:teuthology.orchestra.run.smithi078.stdout:Total 36 MB/s | 99 MB 00:02 2024-02-13T14:03:12.587 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-02-13T14:03:12.952 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-02-13T14:03:12.952 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-02-13T14:03:13.572 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-02-13T14:03:14.017 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-02-13T14:03:14.017 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-02-13T14:03:14.211 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:14.337 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:15.472 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:15.650 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:16.390 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-02-13T14:03:17.109 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:17.134 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:17.268 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:17.347 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:20.059 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:20.254 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:20.704 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:20.813 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:22.018 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:22.220 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:25.093 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:25.898 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:26.964 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:29.834 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.003 INFO:teuthology.orchestra.run.smithi170.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-13T14:03:56.003 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:56.075 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.527 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.527 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:56.527 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:56.528 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:56.690 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.690 INFO:teuthology.orchestra.run.smithi055.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-13T14:03:56.690 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:56.763 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.958 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:56.958 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:56.959 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-02-13T14:03:57.170 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:57.171 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:57.171 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:57.171 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:57.451 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:03:57.575 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.575 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:03:57.576 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-13T14:03:57.576 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T14:03:57.576 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T14:03:57.576 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-13T14:03:57.576 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi170.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T14:03:57.576 DEBUG:teuthology.orchestra.run.smithi170:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:57.641 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-02-13T14:03:57.683 INFO:teuthology.orchestra.run.smithi170.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:57.684 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:03:57.684 DEBUG:teuthology.orchestra.run.smithi170:> sudo rpm -qi grub2-tools 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Name : grub2-tools 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Epoch : 1 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Version : 2.06 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Release : 68.el9 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Architecture: x86_64 2024-02-13T14:03:57.718 INFO:teuthology.orchestra.run.smithi170.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Group : Unspecified 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Size : 8264254 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:License : GPLv3+ 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Packager : builder@centos.org 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Vendor : CentOS 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Summary : Support tools for GRUB. 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:Description : 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-13T14:03:57.719 INFO:teuthology.orchestra.run.smithi170.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-13T14:03:57.720 INFO:teuthology.orchestra.run.smithi170.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-13T14:03:57.720 INFO:teuthology.orchestra.run.smithi170.stdout:hardware devices. 2024-02-13T14:03:57.720 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:03:57.720 INFO:teuthology.orchestra.run.smithi170.stdout:This subpackage provides tools for support of all platforms. 2024-02-13T14:03:57.721 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-13T14:03:57.721 INFO:teuthology.task.kernel:Updating grub on smithi170 to boot 5.14.0-419.el9.x86_64 2024-02-13T14:03:57.721 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-13T14:03:58.124 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:03:58.308 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:58.309 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:03:58.309 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-13T14:03:58.309 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T14:03:58.309 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T14:03:58.309 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-13T14:03:58.309 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi055.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T14:03:58.309 DEBUG:teuthology.orchestra.run.smithi055:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:03:58.358 INFO:teuthology.orchestra.run.smithi170.stderr:Generating grub configuration file ... 2024-02-13T14:03:58.422 INFO:teuthology.orchestra.run.smithi055.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:58.422 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:03:58.422 DEBUG:teuthology.orchestra.run.smithi055:> sudo rpm -qi grub2-tools 2024-02-13T14:03:58.461 INFO:teuthology.orchestra.run.smithi055.stdout:Name : grub2-tools 2024-02-13T14:03:58.461 INFO:teuthology.orchestra.run.smithi055.stdout:Epoch : 1 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Version : 2.06 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Release : 68.el9 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Architecture: x86_64 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Group : Unspecified 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Size : 8264254 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:License : GPLv3+ 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Packager : builder@centos.org 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Vendor : CentOS 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Summary : Support tools for GRUB. 2024-02-13T14:03:58.462 INFO:teuthology.orchestra.run.smithi055.stdout:Description : 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout:hardware devices. 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:03:58.463 INFO:teuthology.orchestra.run.smithi055.stdout:This subpackage provides tools for support of all platforms. 2024-02-13T14:03:58.463 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-13T14:03:58.463 INFO:teuthology.task.kernel:Updating grub on smithi055 to boot 5.14.0-419.el9.x86_64 2024-02-13T14:03:58.463 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-13T14:03:59.100 INFO:teuthology.orchestra.run.smithi055.stderr:Generating grub configuration file ... 2024-02-13T14:03:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:59.114 INFO:teuthology.orchestra.run.smithi078.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-13T14:03:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:03:59.184 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:59.586 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:59.586 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-13T14:03:59.586 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-13T14:03:59.586 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-13T14:03:59.667 INFO:teuthology.orchestra.run.smithi170.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-13T14:03:59.719 INFO:teuthology.orchestra.run.smithi170.stderr:done 2024-02-13T14:03:59.722 DEBUG:teuthology.orchestra.run.smithi170:> mktemp 2024-02-13T14:03:59.737 INFO:teuthology.orchestra.run.smithi170.stdout:/tmp/tmp.Z6h1u6x1PP 2024-02-13T14:03:59.738 DEBUG:teuthology.orchestra.run.smithi170:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Z6h1u6x1PP 2024-02-13T14:03:59.808 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0666 /tmp/tmp.Z6h1u6x1PP 2024-02-13T14:03:59.950 DEBUG:teuthology.orchestra.remote:smithi170:/tmp/tmp.Z6h1u6x1PP is 6KB 2024-02-13T14:03:59.991 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-13T14:03:59.991 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:03:59.991 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:03:59.992 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-02-13T14:03:59.998 DEBUG:teuthology.orchestra.run.smithi170:> rm -fr /tmp/tmp.Z6h1u6x1PP 2024-02-13T14:04:00.014 DEBUG:teuthology.orchestra.run.smithi170:> sudo /bin/ls /boot/loader/entries || true 2024-02-13T14:04:00.078 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-13T14:04:00.078 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-13T14:04:00.078 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-13T14:04:00.078 INFO:teuthology.orchestra.run.smithi170.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-13T14:04:00.079 DEBUG:teuthology.orchestra.run.smithi170:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-13T14:04:00.223 DEBUG:teuthology.orchestra.run.smithi170:> sudo shutdown -r now 2024-02-13T14:04:00.417 INFO:teuthology.orchestra.run.smithi055.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-13T14:04:00.457 INFO:teuthology.orchestra.run.smithi055.stderr:done 2024-02-13T14:04:00.462 DEBUG:teuthology.orchestra.run.smithi055:> mktemp 2024-02-13T14:04:00.479 INFO:teuthology.orchestra.run.smithi055.stdout:/tmp/tmp.k1G9feWeAk 2024-02-13T14:04:00.479 DEBUG:teuthology.orchestra.run.smithi055:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.k1G9feWeAk 2024-02-13T14:04:00.522 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:04:00.553 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0666 /tmp/tmp.k1G9feWeAk 2024-02-13T14:04:00.657 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:04:00.658 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:04:00.658 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-13T14:04:00.658 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-13T14:04:00.658 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-13T14:04:00.658 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-13T14:04:00.658 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi078.front.sepia.ceph.com, path=None, version=distro) 2024-02-13T14:04:00.658 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-02-13T14:04:00.694 DEBUG:teuthology.orchestra.remote:smithi055:/tmp/tmp.k1G9feWeAk is 6KB 2024-02-13T14:04:00.744 DEBUG:teuthology.orchestra.run.smithi055:> rm -fr /tmp/tmp.k1G9feWeAk 2024-02-13T14:04:00.758 DEBUG:teuthology.orchestra.run.smithi055:> sudo /bin/ls /boot/loader/entries || true 2024-02-13T14:04:00.766 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-13T14:04:00.767 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-13T14:04:00.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo rpm -qi grub2-tools 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Name : grub2-tools 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Epoch : 1 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Version : 2.06 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Release : 68.el9 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Architecture: x86_64 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Group : Unspecified 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Size : 8264254 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:License : GPLv3+ 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-13T14:04:00.810 INFO:teuthology.orchestra.run.smithi078.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:Packager : builder@centos.org 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:Vendor : CentOS 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:Summary : Support tools for GRUB. 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:Description : 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-13T14:04:00.811 INFO:teuthology.orchestra.run.smithi078.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-13T14:04:00.812 INFO:teuthology.orchestra.run.smithi078.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-13T14:04:00.812 INFO:teuthology.orchestra.run.smithi078.stdout:hardware devices. 2024-02-13T14:04:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:04:00.812 INFO:teuthology.orchestra.run.smithi078.stdout:This subpackage provides tools for support of all platforms. 2024-02-13T14:04:00.813 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-13T14:04:00.813 INFO:teuthology.task.kernel:Updating grub on smithi078 to boot 5.14.0-419.el9.x86_64 2024-02-13T14:04:00.813 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-13T14:04:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-13T14:04:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-13T14:04:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-13T14:04:00.827 INFO:teuthology.orchestra.run.smithi055.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-13T14:04:00.828 DEBUG:teuthology.orchestra.run.smithi055:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-13T14:04:00.982 DEBUG:teuthology.orchestra.run.smithi055:> sudo shutdown -r now 2024-02-13T14:04:01.443 INFO:teuthology.orchestra.run.smithi078.stderr:Generating grub configuration file ... 2024-02-13T14:04:02.852 INFO:teuthology.orchestra.run.smithi078.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-13T14:04:02.890 INFO:teuthology.orchestra.run.smithi078.stderr:done 2024-02-13T14:04:02.892 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-02-13T14:04:02.908 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.kaizHCgJfx 2024-02-13T14:04:02.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.kaizHCgJfx 2024-02-13T14:04:02.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.kaizHCgJfx 2024-02-13T14:04:03.120 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.kaizHCgJfx is 6KB 2024-02-13T14:04:03.169 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.kaizHCgJfx 2024-02-13T14:04:03.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo /bin/ls /boot/loader/entries || true 2024-02-13T14:04:03.248 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-13T14:04:03.248 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-13T14:04:03.249 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-13T14:04:03.249 INFO:teuthology.orchestra.run.smithi078.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-13T14:04:03.250 DEBUG:teuthology.orchestra.run.smithi078:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-13T14:04:03.390 DEBUG:teuthology.orchestra.run.smithi078:> sudo shutdown -r now 2024-02-13T14:04:30.248 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-13T14:04:30.248 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:04:30.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:04:30.986 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-13T14:04:30.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:04:30.986 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:04:33.394 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-13T14:04:33.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:04:33.394 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:04:48.572 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-02-13T14:04:49.344 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-02-13T14:04:51.904 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-02-13T14:04:57.576 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:04:57.577 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:04:58.345 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:04:58.346 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:00.640 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.170 2024-02-13T14:05:00.905 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:05:00.906 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:03.968 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-02-13T14:05:12.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:05:12.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:15.972 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:05:15.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:16.732 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.55 2024-02-13T14:05:28.744 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:05:28.745 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:31.888 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-02-13T14:05:32.345 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:05:32.345 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-13T14:05:32.345 DEBUG:teuthology.orchestra.run.smithi055:> uname -r 2024-02-13T14:05:32.361 INFO:teuthology.orchestra.run.smithi055.stdout:5.14.0-419.el9.x86_64 2024-02-13T14:05:32.362 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-13T14:05:32.362 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T14:05:32.362 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-13T14:05:33.362 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-13T14:05:33.363 DEBUG:teuthology.orchestra.run.smithi055:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T14:05:33.466 INFO:teuthology.orchestra.run.smithi055.stdout:ttyS1 2024-02-13T14:05:33.514 DEBUG:teuthology.parallel:result is None 2024-02-13T14:05:34.396 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.78 2024-02-13T14:05:44.452 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-02-13T14:05:44.874 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:05:44.874 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-419.el9.x86_64"... 2024-02-13T14:05:44.874 DEBUG:teuthology.orchestra.run.smithi170:> uname -r 2024-02-13T14:05:44.888 INFO:teuthology.orchestra.run.smithi170.stdout:5.14.0-419.el9.x86_64 2024-02-13T14:05:44.888 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-13T14:05:44.888 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T14:05:44.888 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-13T14:05:45.889 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-13T14:05:45.890 DEBUG:teuthology.orchestra.run.smithi170:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T14:05:45.985 INFO:teuthology.orchestra.run.smithi170.stdout:ttyS1 2024-02-13T14:05:46.016 DEBUG:teuthology.parallel:result is None 2024-02-13T14:05:49.397 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:05:49.398 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:05:49.971 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-02-13T14:05:50.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:05:50.489 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-13T14:05:50.490 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-02-13T14:05:50.516 INFO:teuthology.orchestra.run.smithi078.stdout:5.14.0-419.el9.x86_64 2024-02-13T14:05:50.517 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-13T14:05:50.517 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-13T14:05:50.517 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-13T14:05:51.517 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-13T14:05:51.517 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-13T14:05:51.565 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2024-02-13T14:05:51.740 DEBUG:teuthology.parallel:result is None 2024-02-13T14:05:51.740 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-13T14:05:51.749 INFO:teuthology.task.internal:Creating test directory... 2024-02-13T14:05:51.749 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T14:05:51.754 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T14:05:51.756 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-13T14:05:51.773 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-13T14:05:51.782 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-13T14:05:51.793 INFO:teuthology.task.internal:Creating archive directory... 2024-02-13T14:05:51.793 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T14:05:51.814 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T14:05:51.816 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-13T14:05:51.955 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-13T14:05:51.963 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-13T14:05:51.963 DEBUG:teuthology.orchestra.run.smithi055:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T14:05:51.965 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T14:05:51.967 DEBUG:teuthology.orchestra.run.smithi170:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-13T14:05:51.994 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:51.997 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:52.005 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:52.007 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:52.011 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:52.019 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-13T14:05:52.020 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-13T14:05:52.027 INFO:teuthology.task.internal:Configuring sudo... 2024-02-13T14:05:52.028 DEBUG:teuthology.orchestra.run.smithi055:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T14:05:52.056 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T14:05:52.058 DEBUG:teuthology.orchestra.run.smithi170:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-13T14:05:52.090 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-13T14:05:52.100 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-13T14:05:52.101 DEBUG:teuthology.orchestra.run.smithi055:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T14:05:52.127 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T14:05:52.129 DEBUG:teuthology.orchestra.run.smithi170:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-13T14:05:52.147 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.224 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.285 DEBUG:teuthology.orchestra.run.smithi055:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.329 DEBUG:teuthology.orchestra.run.smithi055:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.372 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:05:52.372 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T14:05:52.439 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.529 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.653 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:05:52.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T14:05:52.720 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.764 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-13T14:05:52.829 DEBUG:teuthology.orchestra.run.smithi170:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.873 DEBUG:teuthology.orchestra.run.smithi170:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-13T14:05:52.916 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:05:52.917 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-13T14:05:52.985 DEBUG:teuthology.orchestra.run.smithi055:> sudo service rsyslog restart 2024-02-13T14:05:52.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-02-13T14:05:52.989 DEBUG:teuthology.orchestra.run.smithi170:> sudo service rsyslog restart 2024-02-13T14:05:53.027 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:05:53.044 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:05:53.080 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:05:53.487 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-13T14:05:53.495 INFO:teuthology.task.internal:Starting timer... 2024-02-13T14:05:53.495 INFO:teuthology.run_tasks:Running task pcp... 2024-02-13T14:05:53.513 INFO:teuthology.run_tasks:Running task selinux... 2024-02-13T14:05:53.526 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-13T14:05:53.527 DEBUG:teuthology.orchestra.run.smithi055:> sudo service auditd rotate 2024-02-13T14:05:53.588 INFO:teuthology.orchestra.run.smithi055.stdout:Rotating logs: 2024-02-13T14:05:53.589 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-02-13T14:05:53.639 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-02-13T14:05:53.641 DEBUG:teuthology.orchestra.run.smithi170:> sudo service auditd rotate 2024-02-13T14:05:53.702 INFO:teuthology.orchestra.run.smithi170.stdout:Rotating logs: 2024-02-13T14:05:53.704 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-13T14:05:53.704 DEBUG:teuthology.orchestra.run.smithi055:> /usr/sbin/getenforce 2024-02-13T14:05:53.723 INFO:teuthology.orchestra.run.smithi055.stdout:Permissive 2024-02-13T14:05:53.723 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-02-13T14:05:53.749 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-02-13T14:05:53.750 DEBUG:teuthology.orchestra.run.smithi170:> /usr/sbin/getenforce 2024-02-13T14:05:53.777 INFO:teuthology.orchestra.run.smithi170.stdout:Permissive 2024-02-13T14:05:53.777 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi055.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi170.front.sepia.ceph.com': 'permissive'} 2024-02-13T14:05:53.778 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:05:53.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:05:53.807 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:05:53.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:05:53.834 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:05:53.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:05:53.863 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-13T14:05:53.864 DEBUG:teuthology.orchestra.run.smithi055:> sudo /usr/sbin/setenforce permissive 2024-02-13T14:05:53.900 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-02-13T14:05:53.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo /usr/sbin/setenforce permissive 2024-02-13T14:05:53.979 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-13T14:05:53.990 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-13T14:05:54.006 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-13T14:05:54.008 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi055.front.sepia.ceph.com,smithi078.front.sepia.ceph.com,smithi170.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-13T14:10:10.068 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi055.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com'), Remote(name='ubuntu@smithi170.front.sepia.ceph.com')] 2024-02-13T14:10:10.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:10:10.070 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi055.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:10:10.163 DEBUG:teuthology.orchestra.run.smithi055:> true 2024-02-13T14:10:10.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi055.front.sepia.ceph.com' 2024-02-13T14:10:10.246 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:10:10.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:10:10.335 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-02-13T14:10:10.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-02-13T14:10:10.411 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:10:10.412 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi170.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-13T14:10:10.497 DEBUG:teuthology.orchestra.run.smithi170:> true 2024-02-13T14:10:10.571 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi170.front.sepia.ceph.com' 2024-02-13T14:10:10.571 INFO:teuthology.run_tasks:Running task clock... 2024-02-13T14:10:10.581 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-13T14:10:10.581 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T14:10:10.582 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:10:10.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T14:10:10.584 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:10:10.586 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-13T14:10:10.587 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:10:10.614 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-13T14:10:10.615 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-13T14:10:10.631 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-13T14:10:10.632 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-13T14:10:10.645 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-13T14:10:10.661 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-02-13T14:10:10.661 INFO:teuthology.orchestra.run.smithi055.stderr:sudo: ntpd: command not found 2024-02-13T14:10:10.662 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-13T14:10:10.674 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-02-13T14:10:10.675 INFO:teuthology.orchestra.run.smithi055.stdout:506 Cannot talk to daemon 2024-02-13T14:10:10.691 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-13T14:10:10.692 INFO:teuthology.orchestra.run.smithi170.stderr:sudo: ntpd: command not found 2024-02-13T14:10:10.692 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-13T14:10:10.705 INFO:teuthology.orchestra.run.smithi170.stdout:506 Cannot talk to daemon 2024-02-13T14:10:10.707 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-13T14:10:10.708 INFO:teuthology.orchestra.run.smithi055.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-13T14:10:10.722 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-13T14:10:10.738 INFO:teuthology.orchestra.run.smithi170.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-13T14:10:10.760 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.764 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.770 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:10:10.773 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:10:10.773 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-02-13T14:10:10.773 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.773 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.774 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.774 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.794 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.798 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:10:10.799 INFO:teuthology.run_tasks:Running task pexec... 2024-02-13T14:10:10.808 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-13T14:10:10.808 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi055.front.sepia.ceph.com 2024-02-13T14:10:10.809 DEBUG:teuthology.orchestra.run.smithi055:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-13T14:10:10.809 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi078.front.sepia.ceph.com 2024-02-13T14:10:10.809 DEBUG:teuthology.orchestra.run.smithi078:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-13T14:10:10.809 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi170.front.sepia.ceph.com 2024-02-13T14:10:10.809 DEBUG:teuthology.orchestra.run.smithi170:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-13T14:10:11.416 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:52 ago on Tue 13 Feb 2024 02:09:19 PM UTC. 2024-02-13T14:10:11.432 INFO:teuthology.orchestra.run.smithi055.stdout:Last metadata expiration check: 0:00:49 ago on Tue 13 Feb 2024 02:09:22 PM UTC. 2024-02-13T14:10:11.452 INFO:teuthology.orchestra.run.smithi170.stdout:Last metadata expiration check: 0:00:52 ago on Tue 13 Feb 2024 02:09:19 PM UTC. 2024-02-13T14:10:11.570 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-02-13T14:10:11.580 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 3.1 M 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 9.8 M 2024-02-13T14:10:11.581 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-02-13T14:10:11.593 INFO:teuthology.orchestra.run.smithi055.stdout:Dependencies resolved. 2024-02-13T14:10:11.608 INFO:teuthology.orchestra.run.smithi170.stdout:Dependencies resolved. 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout: Package Architecture Version Repository Size 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout:Installing: 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-13T14:10:11.610 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction Summary 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:================================================================================ 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Install 1 Package 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Total download size: 3.1 M 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Installed size: 9.8 M 2024-02-13T14:10:11.611 INFO:teuthology.orchestra.run.smithi170.stdout:Downloading Packages: 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout: Package Architecture Version Repository Size 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:Installing: 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction Summary 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:================================================================================ 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:Install 1 Package 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:10:11.612 INFO:teuthology.orchestra.run.smithi055.stdout:Total download size: 3.1 M 2024-02-13T14:10:11.613 INFO:teuthology.orchestra.run.smithi055.stdout:Installed size: 9.8 M 2024-02-13T14:10:11.613 INFO:teuthology.orchestra.run.smithi055.stdout:Downloading Packages: 2024-02-13T14:10:12.200 INFO:teuthology.orchestra.run.smithi170.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.3 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.201 INFO:teuthology.orchestra.run.smithi170.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:10:12.201 INFO:teuthology.orchestra.run.smithi170.stdout:Total 5.3 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.201 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction check 2024-02-13T14:10:12.204 INFO:teuthology.orchestra.run.smithi078.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.0 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.204 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:10:12.205 INFO:teuthology.orchestra.run.smithi078.stdout:Total 5.0 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.205 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-02-13T14:10:12.207 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction check succeeded. 2024-02-13T14:10:12.207 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction test 2024-02-13T14:10:12.210 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-02-13T14:10:12.210 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-02-13T14:10:12.221 INFO:teuthology.orchestra.run.smithi055.stdout:runc-1.1.12-1.el9.x86_64.rpm 5.1 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.222 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:10:12.222 INFO:teuthology.orchestra.run.smithi055.stdout:Total 5.1 MB/s | 3.1 MB 00:00 2024-02-13T14:10:12.222 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction check 2024-02-13T14:10:12.229 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction check succeeded. 2024-02-13T14:10:12.229 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction test 2024-02-13T14:10:12.278 INFO:teuthology.orchestra.run.smithi170.stdout:Transaction test succeeded. 2024-02-13T14:10:12.279 INFO:teuthology.orchestra.run.smithi170.stdout:Running transaction 2024-02-13T14:10:12.281 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-02-13T14:10:12.282 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-02-13T14:10:12.305 INFO:teuthology.orchestra.run.smithi055.stdout:Transaction test succeeded. 2024-02-13T14:10:12.306 INFO:teuthology.orchestra.run.smithi055.stdout:Running transaction 2024-02-13T14:10:12.475 INFO:teuthology.orchestra.run.smithi170.stdout: Preparing : 1/1 2024-02-13T14:10:12.477 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-02-13T14:10:12.526 INFO:teuthology.orchestra.run.smithi055.stdout: Preparing : 1/1 2024-02-13T14:10:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:12.597 INFO:teuthology.orchestra.run.smithi170.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:12.626 INFO:teuthology.orchestra.run.smithi055.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.156 INFO:teuthology.orchestra.run.smithi170.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.212 INFO:teuthology.orchestra.run.smithi055.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-02-13T14:10:13.472 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-02-13T14:10:13.482 INFO:teuthology.orchestra.run.smithi170.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.483 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:10:13.483 INFO:teuthology.orchestra.run.smithi170.stdout:Installed: 2024-02-13T14:10:13.483 INFO:teuthology.orchestra.run.smithi170.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-13T14:10:13.483 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:10:13.483 INFO:teuthology.orchestra.run.smithi170.stdout:Complete! 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout:Installed: 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:10:13.599 INFO:teuthology.orchestra.run.smithi055.stdout:Complete! 2024-02-13T14:10:13.709 DEBUG:teuthology.parallel:result is None 2024-02-13T14:10:13.765 DEBUG:teuthology.parallel:result is None 2024-02-13T14:10:13.853 DEBUG:teuthology.parallel:result is None 2024-02-13T14:10:13.854 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-13T14:10:13.872 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-13T14:10:13.872 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:10:13.872 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:13.887 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T14:10:13.887 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_1 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:13.941 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:10:09.061990726 +0000 2024-02-13T14:10:13.942 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:09:47.769420015 +0000 2024-02-13T14:10:13.942 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:09:47.769420015 +0000 2024-02-13T14:10:13.942 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:09:47.769420015 +0000 2024-02-13T14:10:13.942 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T14:10:14.006 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:10:14.006 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:10:14.006 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000293014 s, 1.7 MB/s 2024-02-13T14:10:14.007 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T14:10:14.062 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_2 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:10:09.061990726 +0000 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:09:48.113413077 +0000 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:09:48.113413077 +0000 2024-02-13T14:10:14.117 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:09:48.113413077 +0000 2024-02-13T14:10:14.118 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T14:10:14.182 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:10:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:10:14.183 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000275616 s, 1.9 MB/s 2024-02-13T14:10:14.184 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T14:10:14.239 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_3 2024-02-13T14:10:14.293 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:10:09.061990726 +0000 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:09:48.452406241 +0000 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:09:48.452406241 +0000 2024-02-13T14:10:14.294 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:09:48.452406241 +0000 2024-02-13T14:10:14.294 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T14:10:14.359 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:10:14.359 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:10:14.359 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000258505 s, 2.0 MB/s 2024-02-13T14:10:14.360 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T14:10:14.416 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/vg_nvme/lv_4 2024-02-13T14:10:14.470 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T14:10:14.470 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:14.470 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-02-13T14:10:14.470 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:14.470 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:14.471 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:10:09.061990726 +0000 2024-02-13T14:10:14.471 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:09:48.814398941 +0000 2024-02-13T14:10:14.471 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:09:48.814398941 +0000 2024-02-13T14:10:14.471 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:09:48.814398941 +0000 2024-02-13T14:10:14.471 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T14:10:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:10:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:10:14.545 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000331866 s, 1.5 MB/s 2024-02-13T14:10:14.546 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T14:10:14.604 DEBUG:teuthology.orchestra.run.smithi055:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-13T14:10:14.769 INFO:teuthology.orchestra.run.smithi055.stdout:loop 2024-02-13T14:10:14.770 INFO:tasks.nvme_loop:Connecting nvme_loop smithi055:/dev/vg_nvme/lv_1... 2024-02-13T14:10:14.771 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-13T14:10:14.808 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-02-13T14:10:14.845 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_11 2024-02-13T14:10:14.882 INFO:tasks.nvme_loop:Connecting nvme_loop smithi055:/dev/vg_nvme/lv_2... 2024-02-13T14:10:14.882 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-13T14:10:14.962 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-02-13T14:10:14.999 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_21 2024-02-13T14:10:15.027 INFO:tasks.nvme_loop:Connecting nvme_loop smithi055:/dev/vg_nvme/lv_3... 2024-02-13T14:10:15.027 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-13T14:10:15.112 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-02-13T14:10:15.150 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_31 2024-02-13T14:10:15.180 INFO:tasks.nvme_loop:Connecting nvme_loop smithi055:/dev/vg_nvme/lv_4... 2024-02-13T14:10:15.180 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-13T14:10:15.258 INFO:teuthology.orchestra.run.smithi055.stdout:1 2024-02-13T14:10:15.297 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/vg_nvme/lv_41 2024-02-13T14:10:15.327 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:10:15.327 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:15.382 DEBUG:teuthology.orchestra.run.smithi055:> sudo nvme list 2024-02-13T14:10:15.448 INFO:teuthology.orchestra.run.smithi055.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-13T14:10:15.448 INFO:teuthology.orchestra.run.smithi055.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-13T14:10:15.448 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/nvme0n1 BTLE736102VS4P0KGN INTEL SSDPEDKE040T7 1 4.00 TB / 4.00 TB 512 B + 0 B QDV1013D 2024-02-13T14:10:15.449 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/nvme1n1 1703b7d952667888231c Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:15.449 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/nvme2n1 0b05971227b046f88a4f Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:15.449 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/nvme3n1 0ab65186b915a0c8cc2a Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:15.449 INFO:teuthology.orchestra.run.smithi055.stdout:/dev/nvme4n1 97dd25e9fc01671a7000 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:15.450 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:10:15.450 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:10:15.450 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/scratch_devs 2024-02-13T14:10:15.516 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:10:15.517 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:15.535 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T14:10:15.535 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:10:09.036620508 +0000 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:09:43.749033105 +0000 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:09:43.749033105 +0000 2024-02-13T14:10:15.593 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:09:43.749033105 +0000 2024-02-13T14:10:15.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T14:10:15.663 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:10:15.663 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:10:15.663 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000262877 s, 1.9 MB/s 2024-02-13T14:10:15.664 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T14:10:15.724 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-02-13T14:10:15.782 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T14:10:15.782 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:15.782 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-02-13T14:10:15.782 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:15.782 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:15.783 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:10:09.036620508 +0000 2024-02-13T14:10:15.783 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:09:44.097027429 +0000 2024-02-13T14:10:15.783 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:09:44.097027429 +0000 2024-02-13T14:10:15.783 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:09:44.097027429 +0000 2024-02-13T14:10:15.783 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T14:10:15.852 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:10:15.853 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:10:15.854 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.00021829 s, 2.3 MB/s 2024-02-13T14:10:15.855 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T14:10:15.914 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-02-13T14:10:15.971 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:10:09.036620508 +0000 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:09:44.443021786 +0000 2024-02-13T14:10:15.972 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:09:44.443021786 +0000 2024-02-13T14:10:15.973 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:09:44.443021786 +0000 2024-02-13T14:10:15.973 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T14:10:16.041 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:10:16.041 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:10:16.041 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000217915 s, 2.3 MB/s 2024-02-13T14:10:16.042 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T14:10:16.099 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-02-13T14:10:16.156 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:10:09.036620508 +0000 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:09:44.774016387 +0000 2024-02-13T14:10:16.157 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:09:44.774016387 +0000 2024-02-13T14:10:16.158 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:09:44.774016387 +0000 2024-02-13T14:10:16.158 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T14:10:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:10:16.224 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:10:16.225 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000245476 s, 2.1 MB/s 2024-02-13T14:10:16.226 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T14:10:16.283 DEBUG:teuthology.orchestra.run.smithi078:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-13T14:10:16.427 INFO:teuthology.orchestra.run.smithi078.stdout:loop 2024-02-13T14:10:16.429 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_1... 2024-02-13T14:10:16.429 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-13T14:10:16.467 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-13T14:10:16.503 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_11 2024-02-13T14:10:16.533 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_2... 2024-02-13T14:10:16.533 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-13T14:10:16.610 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-13T14:10:16.660 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_21 2024-02-13T14:10:16.689 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_3... 2024-02-13T14:10:16.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-13T14:10:16.767 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-13T14:10:16.805 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_31 2024-02-13T14:10:16.834 INFO:tasks.nvme_loop:Connecting nvme_loop smithi078:/dev/vg_nvme/lv_4... 2024-02-13T14:10:16.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-13T14:10:16.914 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-02-13T14:10:16.953 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/vg_nvme/lv_41 2024-02-13T14:10:16.990 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:10:16.990 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:17.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme list 2024-02-13T14:10:17.112 INFO:teuthology.orchestra.run.smithi078.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-13T14:10:17.112 INFO:teuthology.orchestra.run.smithi078.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-13T14:10:17.112 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme0n1 PHFT620400QR400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-13T14:10:17.113 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme1n1 338a0b4a8d3b14a42642 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:17.113 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme2n1 01255163cc4ddf503269 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:17.113 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme3n1 919fc64e60babb55c756 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:17.113 INFO:teuthology.orchestra.run.smithi078.stdout:/dev/nvme4n1 95f547568fb8f4a65f38 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:17.114 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:10:17.114 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:10:17.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/scratch_devs 2024-02-13T14:10:17.181 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:10:17.181 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:17.200 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-13T14:10:17.200 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_1 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:17.258 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:10:09.043907675 +0000 2024-02-13T14:10:17.259 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:09:46.815429945 +0000 2024-02-13T14:10:17.259 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:09:46.815429945 +0000 2024-02-13T14:10:17.259 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:09:46.815429945 +0000 2024-02-13T14:10:17.259 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-13T14:10:17.329 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:10:17.329 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:10:17.329 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 9.8391e-05 s, 5.2 MB/s 2024-02-13T14:10:17.330 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-13T14:10:17.388 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_2 2024-02-13T14:10:17.445 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:10:09.043907675 +0000 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:09:47.148422106 +0000 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:09:47.148422106 +0000 2024-02-13T14:10:17.446 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:09:47.148422106 +0000 2024-02-13T14:10:17.446 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-13T14:10:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:10:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:10:17.514 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.2842e-05 s, 6.2 MB/s 2024-02-13T14:10:17.515 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-13T14:10:17.573 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_3 2024-02-13T14:10:17.628 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:10:09.043907675 +0000 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:09:47.473414454 +0000 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:09:47.473414454 +0000 2024-02-13T14:10:17.629 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:09:47.473414454 +0000 2024-02-13T14:10:17.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-13T14:10:17.695 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:10:17.695 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:10:17.696 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000108188 s, 4.7 MB/s 2024-02-13T14:10:17.697 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-13T14:10:17.753 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/vg_nvme/lv_4 2024-02-13T14:10:17.808 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-13T14:10:17.808 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-13T14:10:17.808 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-13T14:10:17.808 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-13T14:10:17.808 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:device_t:s0 2024-02-13T14:10:17.809 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:10:09.043907675 +0000 2024-02-13T14:10:17.809 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:09:47.803406685 +0000 2024-02-13T14:10:17.809 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:09:47.803406685 +0000 2024-02-13T14:10:17.809 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:09:47.803406685 +0000 2024-02-13T14:10:17.809 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-13T14:10:17.875 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:10:17.875 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:10:17.876 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 8.5498e-05 s, 6.0 MB/s 2024-02-13T14:10:17.877 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-13T14:10:17.933 DEBUG:teuthology.orchestra.run.smithi170:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-13T14:10:18.113 INFO:teuthology.orchestra.run.smithi170.stdout:loop 2024-02-13T14:10:18.115 INFO:tasks.nvme_loop:Connecting nvme_loop smithi170:/dev/vg_nvme/lv_1... 2024-02-13T14:10:18.115 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-13T14:10:18.152 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-02-13T14:10:18.190 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/vg_nvme/lv_11 2024-02-13T14:10:18.219 INFO:tasks.nvme_loop:Connecting nvme_loop smithi170:/dev/vg_nvme/lv_2... 2024-02-13T14:10:18.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-13T14:10:18.304 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-02-13T14:10:18.343 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/vg_nvme/lv_21 2024-02-13T14:10:18.373 INFO:tasks.nvme_loop:Connecting nvme_loop smithi170:/dev/vg_nvme/lv_3... 2024-02-13T14:10:18.374 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-13T14:10:18.412 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-02-13T14:10:18.451 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/vg_nvme/lv_31 2024-02-13T14:10:18.480 INFO:tasks.nvme_loop:Connecting nvme_loop smithi170:/dev/vg_nvme/lv_4... 2024-02-13T14:10:18.480 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-13T14:10:18.559 INFO:teuthology.orchestra.run.smithi170.stdout:1 2024-02-13T14:10:18.598 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/vg_nvme/lv_41 2024-02-13T14:10:18.635 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:10:18.635 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:10:18.690 DEBUG:teuthology.orchestra.run.smithi170:> sudo nvme list 2024-02-13T14:10:18.756 INFO:teuthology.orchestra.run.smithi170.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/nvme0n1 PHMB7476002R480DGN INTEL SSDPED1D480GA 1 480.10 GB / 480.10 GB 512 B + 0 B E2010325 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/nvme1n1 a6dd76a1bacd52ba1f6b Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/nvme2n1 83df34e284b725f9b2c6 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/nvme3n1 a77f88c7e758642c7773 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:18.757 INFO:teuthology.orchestra.run.smithi170.stdout:/dev/nvme4n1 55782199e3e9a91978aa Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-13T14:10:18.758 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:10:18.758 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:10:18.758 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/scratch_devs 2024-02-13T14:10:18.824 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-13T14:10:18.927 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662'} 2024-02-13T14:10:18.928 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:10:18.928 INFO:tasks.cephadm:Cluster fsid is 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:10:18.928 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-13T14:10:18.928 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.55', 'mon.b': '172.21.15.78', 'mon.c': '172.21.15.170'} 2024-02-13T14:10:18.929 INFO:tasks.cephadm:First mon is mon.a on smithi055 2024-02-13T14:10:18.929 INFO:tasks.cephadm:First mgr is y 2024-02-13T14:10:18.929 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-13T14:10:18.929 DEBUG:teuthology.orchestra.run.smithi055:> sudo hostname $(hostname -s) 2024-02-13T14:10:18.963 DEBUG:teuthology.orchestra.run.smithi078:> sudo hostname $(hostname -s) 2024-02-13T14:10:18.994 DEBUG:teuthology.orchestra.run.smithi170:> sudo hostname $(hostname -s) 2024-02-13T14:10:19.021 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-13T14:10:19.022 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:10:19.277 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'e1b79d5a4193a484176863ab96c0ca28c00e1662', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76678/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1423-ge1b79d5a', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1423.ge1b79d5a'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-12 23:59:26.532406', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-phlogistonjohn-testing-2024-02-12-1654', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-13T14:10:19.416 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-phlogistonjohn-testing-2024-02-12-1654, sha1 e1b79d5a4193a484176863ab96c0ca28c00e1662 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:10:19.418 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm 2024-02-13T14:10:19.418 DEBUG:teuthology.orchestra.run.smithi055:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:20.036 INFO:teuthology.orchestra.run.smithi055.stdout:-rw-r--r--. 1 ubuntu ubuntu 779193 Feb 13 14:10 /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:20.037 DEBUG:teuthology.orchestra.run.smithi078:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:20.641 INFO:teuthology.orchestra.run.smithi078.stdout:-rw-r--r--. 1 ubuntu ubuntu 779193 Feb 13 14:10 /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:20.641 DEBUG:teuthology.orchestra.run.smithi170:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-phlogistonjohn-testing-2024-02-12-1654/e1b79d5a4193a484176863ab96c0ca28c00e1662/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:21.268 INFO:teuthology.orchestra.run.smithi170.stdout:-rw-r--r--. 1 ubuntu ubuntu 779193 Feb 13 14:10 /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:21.268 DEBUG:teuthology.orchestra.run.smithi055:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:21.287 DEBUG:teuthology.orchestra.run.smithi078:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:21.305 DEBUG:teuthology.orchestra.run.smithi170:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-13T14:10:21.338 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 on all hosts... 2024-02-13T14:10:21.338 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 pull 2024-02-13T14:10:21.340 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 pull 2024-02-13T14:10:21.347 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 pull 2024-02-13T14:10:22.123 INFO:teuthology.orchestra.run.smithi078.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:10:22.186 INFO:teuthology.orchestra.run.smithi055.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:10:22.247 INFO:teuthology.orchestra.run.smithi170.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ceph_version": "ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev)", 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout: "image_id": "28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a", 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout: "repo_digests": [ 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f" 2024-02-13T14:10:49.685 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-02-13T14:10:49.686 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-02-13T14:11:30.293 INFO:teuthology.orchestra.run.smithi170.stdout:{ 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout: "ceph_version": "ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev)", 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout: "image_id": "28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a", 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout: "repo_digests": [ 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f" 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout: ] 2024-02-13T14:11:30.294 INFO:teuthology.orchestra.run.smithi170.stdout:} 2024-02-13T14:11:34.600 INFO:teuthology.orchestra.run.smithi055.stdout:{ 2024-02-13T14:11:34.600 INFO:teuthology.orchestra.run.smithi055.stdout: "ceph_version": "ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev)", 2024-02-13T14:11:34.600 INFO:teuthology.orchestra.run.smithi055.stdout: "image_id": "28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a", 2024-02-13T14:11:34.600 INFO:teuthology.orchestra.run.smithi055.stdout: "repo_digests": [ 2024-02-13T14:11:34.600 INFO:teuthology.orchestra.run.smithi055.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f" 2024-02-13T14:11:34.601 INFO:teuthology.orchestra.run.smithi055.stdout: ] 2024-02-13T14:11:34.601 INFO:teuthology.orchestra.run.smithi055.stdout:} 2024-02-13T14:11:34.628 DEBUG:teuthology.orchestra.run.smithi055:> sudo mkdir -p /etc/ceph 2024-02-13T14:11:34.672 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph 2024-02-13T14:11:34.708 DEBUG:teuthology.orchestra.run.smithi170:> sudo mkdir -p /etc/ceph 2024-02-13T14:11:34.742 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 777 /etc/ceph 2024-02-13T14:11:34.769 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /etc/ceph 2024-02-13T14:11:34.799 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 777 /etc/ceph 2024-02-13T14:11:34.830 INFO:tasks.cephadm:Writing seed config... 2024-02-13T14:11:34.831 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-13T14:11:34.831 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-13T14:11:34.831 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-13T14:11:34.832 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-13T14:11:34.832 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:11:34.833 DEBUG:teuthology.orchestra.run.smithi055:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-13T14:11:34.847 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 9e367004-ca79-11ee-95b9-87774f69a715 mon election default strategy = 1 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-13T14:11:34.848 DEBUG:teuthology.orchestra.run.smithi055:mon.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service 2024-02-13T14:11:34.890 DEBUG:teuthology.orchestra.run.smithi055:mgr.y> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y.service 2024-02-13T14:11:34.933 INFO:tasks.cephadm:Bootstrapping... 2024-02-13T14:11:34.933 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 -v bootstrap --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.55 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:35.129 INFO:teuthology.orchestra.run.smithi055.stdout:-------------------------------------------------------------------------------- 2024-02-13T14:11:35.129 INFO:teuthology.orchestra.run.smithi055.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662', '-v', 'bootstrap', '--fsid', '9e367004-ca79-11ee-95b9-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.55', '--skip-admin-label'] 2024-02-13T14:11:35.154 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:11:35.154 INFO:teuthology.orchestra.run.smithi055.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-13T14:11:35.155 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying podman|docker is present... 2024-02-13T14:11:35.177 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:11:35.177 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying lvm2 is present... 2024-02-13T14:11:35.177 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying time synchronization is in place... 2024-02-13T14:11:35.185 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-13T14:11:35.185 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T14:11:35.190 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-13T14:11:35.190 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:11:35.196 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout enabled 2024-02-13T14:11:35.202 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout active 2024-02-13T14:11:35.202 INFO:teuthology.orchestra.run.smithi055.stdout:Unit chronyd.service is enabled and running 2024-02-13T14:11:35.202 INFO:teuthology.orchestra.run.smithi055.stdout:Repeating the final host check... 2024-02-13T14:11:35.224 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-13T14:11:35.225 INFO:teuthology.orchestra.run.smithi055.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-13T14:11:35.225 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl is present 2024-02-13T14:11:35.225 INFO:teuthology.orchestra.run.smithi055.stdout:lvcreate is present 2024-02-13T14:11:35.231 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-13T14:11:35.231 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-13T14:11:35.237 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-13T14:11:35.237 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:11:35.243 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout enabled 2024-02-13T14:11:35.249 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout active 2024-02-13T14:11:35.249 INFO:teuthology.orchestra.run.smithi055.stdout:Unit chronyd.service is enabled and running 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Host looks OK 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Cluster fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Acquiring lock 139919930601184 on /run/cephadm/9e367004-ca79-11ee-95b9-87774f69a715.lock 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Lock 139919930601184 acquired on /run/cephadm/9e367004-ca79-11ee-95b9-87774f69a715.lock 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying IP 172.21.15.55 port 3300 ... 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying IP 172.21.15.55 port 6789 ... 2024-02-13T14:11:35.250 INFO:teuthology.orchestra.run.smithi055.stdout:Base mon IP(s) is [172.21.15.55:3300, 172.21.15.55:6789], mon addrv is [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2024-02-13T14:11:35.253 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.55 metric 100 2024-02-13T14:11:35.253 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.55 metric 100 2024-02-13T14:11:35.256 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-13T14:11:35.256 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-13T14:11:35.256 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout inet6 fe80::ae1f:6bff:fef8:2606/64 scope link noprefixroute 2024-02-13T14:11:35.258 INFO:teuthology.orchestra.run.smithi055.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-13T14:11:35.259 INFO:teuthology.orchestra.run.smithi055.stdout:Mon IP `172.21.15.55` is in CIDR network `172.21.0.0/20` 2024-02-13T14:11:35.259 INFO:teuthology.orchestra.run.smithi055.stdout:Mon IP `172.21.15.55` is in CIDR network `172.21.0.0/20` 2024-02-13T14:11:35.259 INFO:teuthology.orchestra.run.smithi055.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-13T14:11:35.259 INFO:teuthology.orchestra.run.smithi055.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-13T14:11:35.260 INFO:teuthology.orchestra.run.smithi055.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stdout 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662... 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Getting image source signatures 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying blob sha256:e42eddc1e8b8b944cc016e5274e8a770c05341ed32a5e7a2a56b9220b0323052 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Copying config sha256:28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a 2024-02-13T14:11:35.762 INFO:teuthology.orchestra.run.smithi055.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-13T14:11:36.649 INFO:teuthology.orchestra.run.smithi055.stdout:ceph: stdout ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev) 2024-02-13T14:11:36.649 INFO:teuthology.orchestra.run.smithi055.stdout:Ceph version: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev) 2024-02-13T14:11:36.649 INFO:teuthology.orchestra.run.smithi055.stdout:Extracting ceph user uid/gid from container image... 2024-02-13T14:11:37.629 INFO:teuthology.orchestra.run.smithi055.stdout:stat: stdout 167 167 2024-02-13T14:11:37.629 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial keys... 2024-02-13T14:11:38.584 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQCZeMtlmUQEOxAAPTXnkqleIFXORdD+V9Pauw== 2024-02-13T14:11:39.515 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQCaeMtl0BrYORAANufZaxjiosTteKgq/TCu5w== 2024-02-13T14:11:40.520 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph-authtool: stdout AQCbeMtlP7cxNhAAZXIzYxlnpay00ZU9WEthuA== 2024-02-13T14:11:40.521 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial monmap... 2024-02-13T14:11:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T14:11:41.476 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:monmaptool for a [v2:172.21.15.55:3300,v1:172.21.15.55:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:setting min_mon_release = quincy 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: set fsid to 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:11:41.477 INFO:teuthology.orchestra.run.smithi055.stdout:Creating mon... 2024-02-13T14:11:42.423 INFO:teuthology.orchestra.run.smithi055.stdout:create mon.a on 2024-02-13T14:11:42.781 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-13T14:11:42.958 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target. 2024-02-13T14:11:42.959 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target → /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target. 2024-02-13T14:11:43.204 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a 2024-02-13T14:11:43.204 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service: Unit ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service not loaded. 2024-02-13T14:11:43.379 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target.wants/ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service → /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715@.service. 2024-02-13T14:11:43.713 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 podman[25251]: 2024-02-13 14:11:43.516735868 +0000 UTC m=+0.018097467 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:11:43.888 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:11:43.888 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-02-13T14:11:43.894 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:11:43.894 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:11:43.894 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-02-13T14:11:43.895 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to enable service . firewalld.service is not available 2024-02-13T14:11:43.895 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mon to start... 2024-02-13T14:11:43.895 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mon... 2024-02-13T14:11:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 podman[25251]: 2024-02-13 14:11:43.782149194 +0000 UTC m=+0.283510792 container init 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:11:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 podman[25251]: 2024-02-13 14:11:43.787318641 +0000 UTC m=+0.288680228 container start 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-02-13T14:11:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 ceph-mon[25289]: mkfs 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 ceph-mon[25289]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:11:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 bash[25251]: 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 2024-02-13T14:11:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:43 smithi055 systemd[1]: Started Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: monmap e1: 1 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: fsmap 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: osdmap e1: 0 total, 0 up, 0 in 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: mgrmap e1: no daemons active 2024-02-13T14:11:45.023 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:44 smithi055 ceph-mon[25289]: from='client.? 172.21.15.55:0/3183663154' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-13T14:11:45.026 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout cluster: 2024-02-13T14:11:45.026 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout id: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout services: 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.587025s) 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout data: 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout pgs: 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:45.027 INFO:teuthology.orchestra.run.smithi055.stdout:mon is available 2024-02-13T14:11:45.028 INFO:teuthology.orchestra.run.smithi055.stdout:Assimilating anything we can from ceph.conf... 2024-02-13T14:11:46.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:46.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [global] 2024-02-13T14:11:46.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout fsid = 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:46.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.55:3300,v1:172.21.15.55:6789] 2024-02-13T14:11:46.074 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [osd] 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-13T14:11:46.075 INFO:teuthology.orchestra.run.smithi055.stdout:Generating new minimal ceph.conf... 2024-02-13T14:11:46.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:45 smithi055 ceph-mon[25289]: from='client.? 172.21.15.55:0/3072146489' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T14:11:46.187 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:45 smithi055 ceph-mon[25289]: from='client.? 172.21.15.55:0/3072146489' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-13T14:11:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:46 smithi055 ceph-mon[25289]: from='client.? 172.21.15.55:0/4269253596' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:11:47.087 INFO:teuthology.orchestra.run.smithi055.stdout:Restarting the monitor... 2024-02-13T14:11:47.341 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 systemd[1]: Stopping Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:11:47.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a[25266]: 2024-02-13T14:11:47.338+0000 7f75cbaf8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:11:47.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a[25266]: 2024-02-13T14:11:47.338+0000 7f75cbaf8700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-13T14:11:47.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 podman[25611]: 2024-02-13 14:11:47.585395744 +0000 UTC m=+0.333521335 container died 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-13T14:11:48.120 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 podman[25611]: 2024-02-13 14:11:47.894100164 +0000 UTC m=+0.642225763 container cleanup 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-02-13T14:11:48.121 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:47 smithi055 bash[25611]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a 2024-02-13T14:11:48.471 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 podman[25628]: 2024-02-13 14:11:48.11999075 +0000 UTC m=+0.535575292 container remove 3e5c05e07ca8e6fc97fb3502ee178e5c85019f9aaff11c1ecb3ee69b3c0a1c73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:11:48.472 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-13T14:11:48.472 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 systemd[1]: Stopped Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:48.472 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 systemd[1]: Starting Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:11:48.724 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 podman[25730]: 2024-02-13 14:11:48.470824825 +0000 UTC m=+0.106244256 container create 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-13T14:11:48.724 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 podman[25730]: 2024-02-13 14:11:48.382152798 +0000 UTC m=+0.017572239 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 2024-02-13T14:11:48.903 INFO:teuthology.orchestra.run.smithi055.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-13T14:11:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 podman[25730]: 2024-02-13 14:11:48.806702531 +0000 UTC m=+0.442121960 container init 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-13T14:11:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 podman[25730]: 2024-02-13 14:11:48.812376786 +0000 UTC m=+0.447796214 container start 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=) 2024-02-13T14:11:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:11:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:11:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: pidfile_write: ignore empty --pid-file 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: load: jerasure load: lrc 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Git sha 0 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: DB SUMMARY 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: DB Session ID: BBLLMTPF9SCFQ5J001WL 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84627 ; 2024-02-13T14:11:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.env: 0x55d8a2f5fae0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.info_log: 0x55d8a489b8c0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.statistics: (nil) 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.use_fsync: 0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:11:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.db_log_dir: 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.wal_dir: 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:11:49.068 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.write_buffer_manager: 0x55d8a486f400 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.unordered_write: 0 2024-02-13T14:11:49.069 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:11:49.070 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:11:49.070 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:11:49.070 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:11:49.070 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.row_cache: None 2024-02-13T14:11:49.070 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.wal_filter: None 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.wal_compression: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:11:49.071 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:11:49.072 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_open_files: -1 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Compression algorithms supported: 2024-02-13T14:11:49.073 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kZSTD supported: 0 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.merge_operator: 2024-02-13T14:11:49.074 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_filter: None 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d8a489b9c0) 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_type: 0 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: data_block_index_type: 0 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_shortening: 1 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: checksum: 4 2024-02-13T14:11:49.075 INFO:journalctl@ceph.mon.a.smithi055.stdout: no_block_cache: 0 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache: 0x55d8a489f090 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_options: 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: capacity : 536870912 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: num_shard_bits : 4 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: strict_capacity_limit : 0 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_cache_compressed: (nil) 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: persistent_cache: (nil) 2024-02-13T14:11:49.076 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_size: 4096 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_size_deviation: 10 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_restart_interval: 16 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: index_block_restart_interval: 1 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: metadata_block_size: 4096 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: partition_filters: 0 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: use_delta_encoding: 1 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: filter_policy: bloomfilter 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: whole_key_filtering: 1 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: verify_compression: 0 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: format_version: 5 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: enable_index_compression: 1 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: block_align: 0 2024-02-13T14:11:49.077 INFO:journalctl@ceph.mon.a.smithi055.stdout: max_auto_readahead_size: 262144 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout: prepopulate_block_cache: 0 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression: NoCompression 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.num_levels: 7 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:11:49.078 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:11:49.079 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:11:49.080 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:11:49.081 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:11:49.082 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.ttl: 2592000 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:11:49.083 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: dbef1344-c2e7-41d2-98f5-17688ef40e71 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833508842906, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833508844203, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79439, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73748, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833508, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "dbef1344-c2e7-41d2-98f5-17688ef40e71", "db_session_id": "BBLLMTPF9SCFQ5J001WL", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833508844317, "job": 1, "event": "recovery_finished"} 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d8a4990000 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: DB pointer 0x55d8a497a000 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:11:49.084 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** DB Stats ** 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: L0 2/0 81.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:11:49.085 INFO:journalctl@ceph.mon.a.smithi055.stdout: Sum 2/0 81.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 100.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** Compaction Stats [default] ** 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 100.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:11:49.086 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: Cumulative compaction: 0.00 GB write, 12.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: Interval compaction: 0.00 GB write, 12.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: Block cache BinnedLRUCache@0x55d8a489f090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: starting mon.a rank 0 at public addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] at bind addrs [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???) e1 preinit fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).mds e1 new map 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).mds e1 print_map 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: e1 2024-02-13T14:11:49.087 INFO:journalctl@ceph.mon.a.smithi055.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout: legacy client fscid: -1 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout: 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout: No filesystems configured 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 bash[25730]: 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 systemd[1]: Started Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: monmap e1: 1 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: fsmap 2024-02-13T14:11:49.088 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: osdmap e1: 0 total, 0 up, 0 in 2024-02-13T14:11:49.089 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:48 smithi055 ceph-mon[25769]: mgrmap e1: no daemons active 2024-02-13T14:11:49.975 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-13T14:11:49.996 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:11:49.997 INFO:teuthology.orchestra.run.smithi055.stdout:Creating mgr... 2024-02-13T14:11:49.997 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-13T14:11:49.997 INFO:teuthology.orchestra.run.smithi055.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-13T14:11:50.261 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y 2024-02-13T14:11:50.261 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Failed to reset failed state of unit ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y.service: Unit ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y.service not loaded. 2024-02-13T14:11:50.410 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715.target.wants/ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-9e367004-ca79-11ee-95b9-87774f69a715@.service. 2024-02-13T14:11:50.767 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:50 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3689004796' entity='client.admin' 2024-02-13T14:11:50.768 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:50 smithi055 podman[26009]: 2024-02-13 14:11:50.647823886 +0000 UTC m=+0.120282322 container create 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:11:50.998 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:11:50.998 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-02-13T14:11:51.005 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:11:51.005 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:11:51.005 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-02-13T14:11:51.005 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to enable service . firewalld.service is not available 2024-02-13T14:11:51.013 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:11:51.013 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr to start... 2024-02-13T14:11:51.020 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr... 2024-02-13T14:11:51.144 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:50 smithi055 podman[26009]: 2024-02-13 14:11:50.875721272 +0000 UTC m=+0.348179664 container init 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-02-13T14:11:51.144 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:50 smithi055 podman[26009]: 2024-02-13 14:11:50.881103802 +0000 UTC m=+0.353562196 container start 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-02-13T14:11:51.144 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:50 smithi055 bash[26009]: 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e 2024-02-13T14:11:51.144 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:50 smithi055 systemd[1]: Started Ceph mgr.y for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:11:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:51 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3124921894' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:11:51.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:51.445+0000 7fc1dc72a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:11:51.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:51.543+0000 7fc1dc72a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:11:52.098 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "9e367004-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:11:52.099 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:11:52.100 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:11:52.101 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:11:43.842469+0000", 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:11:52.102 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:11:52.103 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:11:52.103 INFO:teuthology.orchestra.run.smithi055.stdout:mgr not available, waiting (1/15)... 2024-02-13T14:11:52.549 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.214+0000 7fc1dc72a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:11:52.549 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.308+0000 7fc1dc72a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:11:52.549 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.450+0000 7fc1dc72a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:11:52.549 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.547+0000 7fc1dc72a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:11:52.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.632+0000 7fc1dc72a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:11:53.194 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:52 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:52.898+0000 7fc1dc72a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:11:53.194 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:53 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:53.009+0000 7fc1dc72a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:11:53.194 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:53 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:53.096+0000 7fc1dc72a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:11:53.194 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:53 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:53.192+0000 7fc1dc72a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:11:53.564 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:53 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:53.369+0000 7fc1dc72a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:11:54.507 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:54 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:54.372+0000 7fc1dc72a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:11:54.814 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:54 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:54.505+0000 7fc1dc72a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:11:55.310 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:54 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:54.842+0000 7fc1dc72a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:11:55.310 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:54 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:54.933+0000 7fc1dc72a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:11:55.310 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:55 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:55.130+0000 7fc1dc72a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:11:55.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:55 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:55.321+0000 7fc1dc72a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:11:55.877 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:55 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:55.613+0000 7fc1dc72a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:11:55.877 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:55 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:55.875+0000 7fc1dc72a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:11:56.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:56 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:56.116+0000 7fc1dc72a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:11:57.314 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:56 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:56.948+0000 7fc1dc72a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:11:57.639 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:57 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3316469249' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:11:57.639 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:57 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:57.538+0000 7fc1dc72a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:11:57.639 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:57 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:57.637+0000 7fc1dc72a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:11:58.064 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:11:57 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:11:57.857+0000 7fc1dc72a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: Activating manager daemon y 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: mgrmap e2: y(active, starting, since 0.00468053s) 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: Manager daemon y is now available 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:11:58.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:11:58.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' 2024-02-13T14:11:58.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' 2024-02-13T14:11:58.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:58 smithi055 ceph-mon[25769]: from='mgr.14100 172.21.15.55:0/376557437' entity='mgr.y' 2024-02-13T14:12:00.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:11:59 smithi055 ceph-mon[25769]: mgrmap e3: y(active, since 1.00906s) 2024-02-13T14:12:02.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:01 smithi055 ceph-mon[25769]: mgrmap e4: y(active, since 3s) 2024-02-13T14:12:03.427 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:12:03.427 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "9e367004-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:03.428 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:12:03.429 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:12:03.430 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:12:03.430 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:12:03.435 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.435 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:12:03.435 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:12:03.435 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.436 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:12:03.437 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:12:03.438 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:11:43.842469+0000", 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:03.439 INFO:teuthology.orchestra.run.smithi055.stdout:mgr not available, waiting (2/15)... 2024-02-13T14:12:06.564 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:06 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3037806613' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-13T14:12:06.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:12:06.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:06.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsid": "9e367004-ca79-11ee-95b9-87774f69a715", 2024-02-13T14:12:06.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "health": { 2024-02-13T14:12:06.871 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 0 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "a" 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "quorum_age": 17, 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-13T14:12:06.872 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-13T14:12:06.873 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-13T14:12:06.877 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "restful" 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ], 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-13T14:12:06.878 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "modified": "2024-02-13T14:11:43.842469+0000", 2024-02-13T14:12:06.879 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-13T14:12:06.879 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout }, 2024-02-13T14:12:06.879 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-13T14:12:06.879 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:06.879 INFO:teuthology.orchestra.run.smithi055.stdout:mgr is available 2024-02-13T14:12:07.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:07 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/714639480' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-13T14:12:07.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:07 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/714639480' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [global] 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout fsid = 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:12:08.159 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout [osd] 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-13T14:12:08.160 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling cephadm module... 2024-02-13T14:12:09.260 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:08 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4070111284' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-13T14:12:09.260 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:08 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setuser ceph since I am not root 2024-02-13T14:12:09.260 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:08 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setgroup ceph since I am not root 2024-02-13T14:12:09.786 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:09 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:09.521+0000 7f854d2ec200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:12:09.787 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:09 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:09.619+0000 7f854d2ec200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:12:10.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:09 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4070111284' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-13T14:12:10.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:09 smithi055 ceph-mon[25769]: mgrmap e5: y(active, since 11s) 2024-02-13T14:12:10.538 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:10.290+0000 7f854d2ec200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:12:10.538 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:10.390+0000 7f854d2ec200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:12:10.539 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:10.535+0000 7f854d2ec200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:12:10.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:10.638+0000 7f854d2ec200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:12:10.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:10.727+0000 7f854d2ec200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for the mgr to restart... 2024-02-13T14:12:10.913 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr epoch 5... 2024-02-13T14:12:11.220 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:10 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2330575639' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T14:12:11.220 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:11 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:11.013+0000 7f854d2ec200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:12:11.220 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:11 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:11.127+0000 7f854d2ec200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:12:11.220 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:11 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:11.217+0000 7f854d2ec200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:12:11.486 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:11 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:11.321+0000 7f854d2ec200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:12:11.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:11 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:11.504+0000 7f854d2ec200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:12:12.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:12 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:12.487+0000 7f854d2ec200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:12:12.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:12 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:12.619+0000 7f854d2ec200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:12:13.247 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:12 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:12.955+0000 7f854d2ec200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:12:13.247 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:13.046+0000 7f854d2ec200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:12:13.247 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:13.244+0000 7f854d2ec200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:12:13.564 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:13.431+0000 7f854d2ec200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:12:13.982 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:13.719+0000 7f854d2ec200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:12:13.982 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:13.979+0000 7f854d2ec200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:12:14.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:14.219+0000 7f854d2ec200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:12:15.314 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:15 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:15.038+0000 7f854d2ec200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:12:15.937 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:15 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:15.617+0000 7f854d2ec200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:12:15.937 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:15 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:15.717+0000 7f854d2ec200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:12:15.937 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:15 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:15.935+0000 7f854d2ec200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: Active manager daemon y restarted 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: Activating manager daemon y 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: osdmap e2: 0 total, 0 up, 0 in 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: mgrmap e6: y(active, starting, since 0.00526505s) 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: Manager daemon y is now available 2024-02-13T14:12:16.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:16.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:16.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:16.316 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:16 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: Found migration_current of "None". Setting to last migration. 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:17.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:17 smithi055 ceph-mon[25769]: mgrmap e7: y(active, since 1.01168s) 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:mgr epoch 5 is available 2024-02-13T14:12:17.593 INFO:teuthology.orchestra.run.smithi055.stdout:Setting orchestrator backend to cephadm... 2024-02-13T14:12:18.313 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:18 smithi055 ceph-mon[25769]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T14:12:18.313 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:18 smithi055 ceph-mon[25769]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T14:12:18.313 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:18 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:19.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:17] ENGINE Bus STARTING 2024-02-13T14:12:19.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:17] ENGINE Serving on https://172.21.15.55:7150 2024-02-13T14:12:19.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:17] ENGINE Serving on http://172.21.15.55:8765 2024-02-13T14:12:19.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:17] ENGINE Bus STARTED 2024-02-13T14:12:19.249 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: mgrmap e8: y(active, since 2s) 2024-02-13T14:12:19.249 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:19.249 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:19.249 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:19 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:20.076 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-mon[25769]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:20.079 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-13T14:12:20.079 INFO:teuthology.orchestra.run.smithi055.stdout:Generating ssh key... 2024-02-13T14:12:21.282 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: Generating public/private rsa key pair. 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: Your identification has been saved in /tmp/tmprx69p31z/key. 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: Your public key has been saved in /tmp/tmprx69p31z/key.pub. 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: The key fingerprint is: 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: SHA256:v02IVtEwgq8nnE4d45kaKvg7/SaX+OGz4u2aI9CGpqk ceph-9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: The key's randomart image is: 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: +---[RSA 3072]----+ 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: | .. o | 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: | . . + | 2024-02-13T14:12:21.283 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: | . . . | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: | + . | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: | o . =S=. | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: |o.o B *+ . | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: |o= . =.*o o . | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: |+ + B+Bo + | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: |E.o*+@Bo . . | 2024-02-13T14:12:21.284 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:20 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: +----[SHA256]-----+ 2024-02-13T14:12:22.279 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:21 smithi055 ceph-mon[25769]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:22.279 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:21 smithi055 ceph-mon[25769]: Generating ssh key... 2024-02-13T14:12:22.279 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:21 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:22.279 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:21 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:22.833 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:22.833 INFO:teuthology.orchestra.run.smithi055.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:12:22.833 INFO:teuthology.orchestra.run.smithi055.stdout:Adding key to root@localhost authorized_keys... 2024-02-13T14:12:22.834 INFO:teuthology.orchestra.run.smithi055.stdout:Adding host smithi055... 2024-02-13T14:12:23.094 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:23 smithi055 ceph-mon[25769]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:24.104 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:24 smithi055 ceph-mon[25769]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi055", "addr": "172.21.15.55", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:25.314 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:25 smithi055 ceph-mon[25769]: Deploying cephadm binary to smithi055 2024-02-13T14:12:28.607 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Added host 'smithi055' with addr '172.21.15.55' 2024-02-13T14:12:28.607 INFO:teuthology.orchestra.run.smithi055.stdout:Deploying unmanaged mon service... 2024-02-13T14:12:29.218 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:28 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:29.218 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:28 smithi055 ceph-mon[25769]: Added host smithi055 2024-02-13T14:12:29.218 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:28 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:30.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:30 smithi055 ceph-mon[25769]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:30.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:30 smithi055 ceph-mon[25769]: Saving service mon spec with placement count:5 2024-02-13T14:12:30.441 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:30 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:30.947 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-13T14:12:30.947 INFO:teuthology.orchestra.run.smithi055.stdout:Deploying unmanaged mgr service... 2024-02-13T14:12:31.794 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:31 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:31.794 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:31 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:32.622 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-13T14:12:32.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:32 smithi055 ceph-mon[25769]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:32.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:32 smithi055 ceph-mon[25769]: Saving service mgr spec with placement count:2 2024-02-13T14:12:32.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:32 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:34.537 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:34 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2688765901' entity='client.admin' 2024-02-13T14:12:34.537 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:34 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:35.145 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling the dashboard module... 2024-02-13T14:12:35.798 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:35 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3166991299' entity='client.admin' 2024-02-13T14:12:35.798 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:35 smithi055 ceph-mon[25769]: from='mgr.14118 172.21.15.55:0/1895966685' entity='mgr.y' 2024-02-13T14:12:36.792 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:36 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4077723569' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-13T14:12:36.792 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:36 smithi055 ceph-mon[25769]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:12:36.792 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:36 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setuser ceph since I am not root 2024-02-13T14:12:36.792 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:36 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setgroup ceph since I am not root 2024-02-13T14:12:37.387 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:37.114+0000 7f3012f17200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:12:37.387 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:37.211+0000 7f3012f17200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:12:37.722 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4077723569' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-13T14:12:37.722 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-mon[25769]: mgrmap e9: y(active, since 20s) 2024-02-13T14:12:37.983 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:37.885+0000 7f3012f17200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:12:37.984 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:37 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:37.981+0000 7f3012f17200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:12:38.314 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.129+0000 7f3012f17200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:12:38.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.230+0000 7f3012f17200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:12:38.590 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2007383793' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-13T14:12:38.590 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.316+0000 7f3012f17200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:38.788 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for the mgr to restart... 2024-02-13T14:12:38.789 INFO:teuthology.orchestra.run.smithi055.stdout:Waiting for mgr epoch 9... 2024-02-13T14:12:38.893 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.588+0000 7f3012f17200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:12:38.893 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.701+0000 7f3012f17200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:12:38.894 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.791+0000 7f3012f17200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:12:39.171 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:38 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:38.891+0000 7f3012f17200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:12:39.171 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:39 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:39.084+0000 7f3012f17200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:12:40.558 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:40 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:40.082+0000 7f3012f17200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:12:40.558 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:40 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:40.217+0000 7f3012f17200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:12:40.558 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:40 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:40.556+0000 7f3012f17200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:12:40.816 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:40 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:40.648+0000 7f3012f17200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:12:41.184 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:40 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:40.849+0000 7f3012f17200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:12:41.184 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:41 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:41.039+0000 7f3012f17200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:12:41.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:41 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:41.337+0000 7f3012f17200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:12:41.856 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:41 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:41.605+0000 7f3012f17200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:12:41.856 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:41 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:41.854+0000 7f3012f17200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:12:43.065 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:42.689+0000 7f3012f17200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:12:43.564 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:43.275+0000 7f3012f17200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:12:43.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:43.376+0000 7f3012f17200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:12:43.597+0000 7f3012f17200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: Active manager daemon y restarted 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: Activating manager daemon y 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: osdmap e3: 0 total, 0 up, 0 in 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: mgrmap e10: y(active, starting, since 0.00548718s) 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:12:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:12:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:12:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:12:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: Manager daemon y is now available 2024-02-13T14:12:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:44.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:44] ENGINE Bus STARTING 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:44] ENGINE Serving on https://172.21.15.55:7150 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:44] ENGINE Serving on http://172.21.15.55:8765 2024-02-13T14:12:45.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: [13/Feb/2024:14:12:44] ENGINE Bus STARTED 2024-02-13T14:12:45.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:44 smithi055 ceph-mon[25769]: mgrmap e11: y(active, since 1.00637s) 2024-02-13T14:12:45.296 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout { 2024-02-13T14:12:45.296 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-13T14:12:45.296 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-13T14:12:45.297 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout } 2024-02-13T14:12:45.297 INFO:teuthology.orchestra.run.smithi055.stdout:mgr epoch 9 is available 2024-02-13T14:12:45.297 INFO:teuthology.orchestra.run.smithi055.stdout:Generating a dashboard self-signed certificate... 2024-02-13T14:12:45.705 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:45 smithi055 ceph-mon[25769]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-13T14:12:45.705 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:45 smithi055 ceph-mon[25769]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-13T14:12:46.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:46 smithi055 ceph-mon[25769]: mgrmap e12: y(active, since 2s) 2024-02-13T14:12:46.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:46.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:46 smithi055 ceph-mon[25769]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:46.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:46.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:46.819 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-13T14:12:46.820 INFO:teuthology.orchestra.run.smithi055.stdout:Creating initial admin user... 2024-02-13T14:12:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:47 smithi055 ceph-mon[25769]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:12:48.426 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$bzhL4L/s7Et2yr6qlLMBZOpxihZPBiJL5FqkCa5Hoo1F3hogYyvJC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707833567, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-13T14:12:48.427 INFO:teuthology.orchestra.run.smithi055.stdout:Fetching dashboard port number... 2024-02-13T14:12:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:49.760 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stdout 8443 2024-02-13T14:12:49.769 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-13T14:12:49.769 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout disabled 2024-02-13T14:12:49.777 INFO:teuthology.orchestra.run.smithi055.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-13T14:12:49.777 INFO:teuthology.orchestra.run.smithi055.stdout:systemctl: stdout inactive 2024-02-13T14:12:49.777 INFO:teuthology.orchestra.run.smithi055.stdout:firewalld.service is not enabled 2024-02-13T14:12:49.777 INFO:teuthology.orchestra.run.smithi055.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout:Ceph Dashboard is now available at: 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout: URL: https://smithi055.front.sepia.ceph.com:8443/ 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout: User: admin 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout: Password: xqld0huaxv 2024-02-13T14:12:49.779 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:49.780 INFO:teuthology.orchestra.run.smithi055.stdout:Saving cluster configuration to /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config directory 2024-02-13T14:12:49.780 INFO:teuthology.orchestra.run.smithi055.stdout:Enabling autotune for osd_memory_target 2024-02-13T14:12:50.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:49 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4071357214' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-13T14:12:51.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:50 smithi055 ceph-mon[25769]: mgrmap e13: y(active, since 6s) 2024-02-13T14:12:53.244 INFO:teuthology.orchestra.run.smithi055.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-13T14:12:53.244 INFO:teuthology.orchestra.run.smithi055.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-13T14:12:53.244 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.244 INFO:teuthology.orchestra.run.smithi055.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:12:53.244 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout:Or, if you are only running a single cluster on this host: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: ceph telemetry on 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout:For more information see: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:12:53.245 INFO:teuthology.orchestra.run.smithi055.stdout:Bootstrap complete. 2024-02-13T14:12:53.289 INFO:tasks.cephadm:Fetching config... 2024-02-13T14:12:53.289 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:12:53.289 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-13T14:12:53.305 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-13T14:12:53.305 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:12:53.305 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-13T14:12:53.360 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-13T14:12:53.360 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:12:53.360 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-13T14:12:53.427 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-13T14:12:53.427 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:12:53.427 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-13T14:12:53.482 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-13T14:12:53.482 DEBUG:teuthology.orchestra.run.smithi055:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T14:12:53.560 INFO:teuthology.orchestra.run.smithi055.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:53.574 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T14:12:53.617 INFO:teuthology.orchestra.run.smithi078.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:53.632 DEBUG:teuthology.orchestra.run.smithi170:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-13T14:12:53.678 INFO:teuthology.orchestra.run.smithi170.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDZ9a1Gj0gVdQpyVjJWrXSsZb5IRqWfDgdJQPIklEMTHkISuws2vZd1IEvNNG0jLZKdYvd902T5Qt+zb4Yby6KRzMj68tVyGM8Xv4bvwfZgFSrGiAxfLMZkfPxxGhvIG3mm90H3UT7FzGr/b1VSaWQibipWkBTnXxvpSSc2u6QrcXRrdNr2DshtxRPZDqWrbeHuV7ebIU/0jQFCmCoccWwPrRORohzW/EjqAo6qNJQfWCFJrq3P0Q0flTHroSpIKDHWHEW3tyyqSs1Qw5Lls1fHnhJk9ijqb27MunyNQnkcn0rInk1bLe/3d2jWzQa4epquPt0npVYArH8M6RDSF6lj1XUqIiCtD4J9rlIWBIA/dudK9SorQj0CKBFmaRg2HHb4/YPReBcbcQQBncFniaq+o5JIeElDXnDO2g/4RUPoF0bMyDoLPSQ1qzOBhKrg4kIfIkLlWHMswXPjUw8JgwpKq/LSAhwNo8h7CnUmZmJfS1OZezxKz821HLGaWIND4ZM= ceph-9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:12:53.693 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-13T14:12:53.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:53 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3717763686' entity='client.admin' 2024-02-13T14:12:53.935 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:57.427 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-13T14:12:57.427 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-13T14:12:57.991 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:57 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/616921946' entity='client.admin' 2024-02-13T14:12:58.117 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:12:59.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:59.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:59.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi055", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:12:59.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:12:59.248 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:12:59.249 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:12:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:00.752 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi078 2024-02-13T14:13:00.752 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:13:00.753 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.conf 2024-02-13T14:13:00.770 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:13:00.770 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:00.828 INFO:tasks.cephadm:Adding host smithi078 to orchestrator... 2024-02-13T14:13:00.828 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch host add smithi078 2024-02-13T14:13:01.067 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:01.282 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:01.283 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:02.288 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:01 smithi055 ceph-mon[25769]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:04.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:03 smithi055 ceph-mon[25769]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi078", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:05.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:04 smithi055 ceph-mon[25769]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:05.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:04 smithi055 ceph-mon[25769]: Deploying cephadm binary to smithi078 2024-02-13T14:13:07.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:06 smithi055 ceph-mon[25769]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:07.804 INFO:teuthology.orchestra.run.smithi055.stdout:Added host 'smithi078' with addr '172.21.15.78' 2024-02-13T14:13:08.383 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-13T14:13:08.625 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:09.006 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:08 smithi055 ceph-mon[25769]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:09.006 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:09.006 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:08 smithi055 ceph-mon[25769]: Added host smithi078 2024-02-13T14:13:09.006 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:09.006 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:10.033 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:10.524 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:13:10.524 INFO:teuthology.orchestra.run.smithi055.stdout:[{"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}] 2024-02-13T14:13:11.141 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi170 2024-02-13T14:13:11.141 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:13:11.141 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.conf 2024-02-13T14:13:11.161 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:13:11.162 DEBUG:teuthology.orchestra.run.smithi170:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:11.218 INFO:tasks.cephadm:Adding host smithi170 to orchestrator... 2024-02-13T14:13:11.219 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch host add smithi170 2024-02-13T14:13:11.459 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:11.665 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:11 smithi055 ceph-mon[25769]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:11.666 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:11 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:11.666 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:11 smithi055 ceph-mon[25769]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:13.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:13 smithi055 ceph-mon[25769]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:14.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: Deploying cephadm binary to smithi170 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:15 smithi055 ceph-mon[25769]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:17.768 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:17 smithi055 ceph-mon[25769]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:17.768 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:17.768 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:17.768 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:17.768 INFO:teuthology.orchestra.run.smithi055.stdout:Added host 'smithi170' with addr '172.21.15.170' 2024-02-13T14:13:18.380 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch host ls --format=json 2024-02-13T14:13:18.626 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:19.041 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:18 smithi055 ceph-mon[25769]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:19.041 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:18 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:19.041 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:18 smithi055 ceph-mon[25769]: Added host smithi170 2024-02-13T14:13:19.042 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:18 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:19.042 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:18 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:20.059 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:19 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:20.546 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:13:20.546 INFO:teuthology.orchestra.run.smithi055.stdout:[{"addr": "172.21.15.55", "hostname": "smithi055", "labels": [], "status": ""}, {"addr": "172.21.15.78", "hostname": "smithi078", "labels": [], "status": ""}, {"addr": "172.21.15.170", "hostname": "smithi170", "labels": [], "status": ""}] 2024-02-13T14:13:21.197 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-13T14:13:21.197 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd crush tunables default 2024-02-13T14:13:21.437 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:21.559 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:21 smithi055 ceph-mon[25769]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:21.559 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:21 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:21.559 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:21 smithi055 ceph-mon[25769]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:23.397 INFO:teuthology.orchestra.run.smithi055.stderr:adjusted tunables profile to default 2024-02-13T14:13:23.716 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:23 smithi055 ceph-mon[25769]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:23.716 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:23 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:13:24.003 INFO:tasks.cephadm:Adding mon.a on smithi055 2024-02-13T14:13:24.003 INFO:tasks.cephadm:Adding mon.b on smithi078 2024-02-13T14:13:24.003 INFO:tasks.cephadm:Adding mon.c on smithi170 2024-02-13T14:13:24.003 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply mon '3;smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c' 2024-02-13T14:13:24.240 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:24.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:26.129 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mon update... 2024-02-13T14:13:26.725 DEBUG:teuthology.orchestra.run.smithi078:mon.b> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.b.service 2024-02-13T14:13:26.728 DEBUG:teuthology.orchestra.run.smithi170:mon.c> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.c.service 2024-02-13T14:13:26.730 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:13:26.730 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c;count:3 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:27.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:27.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:27.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:27.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:27 smithi055 ceph-mon[25769]: Deploying daemon mon.c on smithi170 2024-02-13T14:13:27.873 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:29.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:29 smithi055 ceph-mon[25769]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:30.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:30 smithi170 systemd[1]: Starting Ceph mon.c for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:13:30.919 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:13:30.919 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":1,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:11:40.903138Z","created":"2024-02-13T14:11:40.903138Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-13T14:13:30.920 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 1 2024-02-13T14:13:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:30 smithi170 podman[30131]: 2024-02-13 14:13:30.756264455 +0000 UTC m=+0.024626989 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:13:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:30 smithi170 podman[30131]: 2024-02-13 14:13:30.859390661 +0000 UTC m=+0.127753188 container create a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-02-13T14:13:31.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:31 smithi055 ceph-mon[25769]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:31 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/1843270271' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:31.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 podman[30131]: 2024-02-13 14:13:31.179260771 +0000 UTC m=+0.447623298 container init a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 podman[30131]: 2024-02-13 14:13:31.184714898 +0000 UTC m=+0.453077428 container start a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pidfile_write: ignore empty --pid-file 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: load: jerasure load: lrc 2024-02-13T14:13:31.631 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Git sha 0 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: DB SUMMARY 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: DB Session ID: WSRIOTBQDJ1SAZFENNKO 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:13:31.632 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:13:31.633 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.env: 0x55c9df4daae0 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.info_log: 0x55c9e08578c0 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:13:31.634 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.statistics: (nil) 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.use_fsync: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:13:31.635 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.db_log_dir: 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.wal_dir: 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.write_buffer_manager: 0x55c9e082b400 2024-02-13T14:13:31.636 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.unordered_write: 0 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.row_cache: None 2024-02-13T14:13:31.637 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.wal_filter: None 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.wal_compression: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:13:31.638 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:13:31.639 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:13:31.639 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:13:31.639 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:13:31.639 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:13:31.639 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_open_files: -1 2024-02-13T14:13:31.640 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Compression algorithms supported: 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kZSTD supported: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:13:31.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.merge_operator: 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_filter: None 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:13:31.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c9e08579c0) 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_type: 0 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_index_type: 0 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_shortening: 1 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: checksum: 4 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: no_block_cache: 0 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache: 0x55c9e085b090 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_options: 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: capacity : 536870912 2024-02-13T14:13:31.643 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_shard_bits : 4 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: strict_capacity_limit : 0 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_cache_compressed: (nil) 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: persistent_cache: (nil) 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size: 4096 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_size_deviation: 10 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_restart_interval: 16 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: index_block_restart_interval: 1 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: metadata_block_size: 4096 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: partition_filters: 0 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: use_delta_encoding: 1 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: filter_policy: bloomfilter 2024-02-13T14:13:31.644 INFO:journalctl@ceph.mon.c.smithi170.stdout: whole_key_filtering: 1 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: verify_compression: 0 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: format_version: 5 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_index_compression: 1 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: block_align: 0 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: max_auto_readahead_size: 262144 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: prepopulate_block_cache: 0 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression: NoCompression 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:13:31.645 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.num_levels: 7 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:13:31.646 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:13:31.647 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:13:31.648 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:13:31.649 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.ttl: 2592000 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:13:31.650 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: cd80433a-c558-4ff7-9ab4-c4c876e83a04 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833611214521, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-13T14:13:31.651 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833611214926, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833611, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "cd80433a-c558-4ff7-9ab4-c4c876e83a04", "db_session_id": "WSRIOTBQDJ1SAZFENNKO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833611215004, "job": 1, "event": "recovery_finished"} 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9e094c000 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: DB pointer 0x55c9e0936000 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** DB Stats ** 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:13:31.652 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** Compaction Stats [default] ** 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:31.653 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Cumulative compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Interval compaction: 0.00 GB write, 0.65 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache BinnedLRUCache@0x55c9e085b090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-02-13T14:13:31.654 INFO:journalctl@ceph.mon.c.smithi170.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: using public_addr v2:172.21.15.170:0/0 -> [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: starting mon.c rank -1 at public addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] at bind addrs [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(???) e0 preinit fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).mds e1 new map 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: e1 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: legacy client fscid: -1 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout: No filesystems configured 2024-02-13T14:13:31.655 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Added host smithi078 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:31.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:31.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Deploying cephadm binary to smithi170 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Added host smithi170 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.659 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:31.660 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c;count:3 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: Deploying daemon mon.c on smithi170 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/1843270271' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 ceph-mon[30181]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 bash[30131]: a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 2024-02-13T14:13:31.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:31 smithi170 systemd[1]: Started Ceph mon.c for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:13:32.663 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:13:32.704 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:13:32.953 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: mon.a calling monitor election 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: mon.c calling monitor election 2024-02-13T14:13:36.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: monmap e2: 2 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: fsmap 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: mgrmap e13: y(active, since 52s) 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: overall HEALTH_OK 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:36.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:36 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: mon.a calling monitor election 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: mon.c calling monitor election 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: monmap e2: 2 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: fsmap 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: mgrmap e13: y(active, since 52s) 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: overall HEALTH_OK 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:36.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:36.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:36 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:37 smithi170 ceph-mon[30181]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:37 smithi170 ceph-mon[30181]: Deploying daemon mon.b on smithi078 2024-02-13T14:13:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:37 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:37.773 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:13:37.773 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":2,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:13:31.238625Z","created":"2024-02-13T14:11:40.903138Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-13T14:13:37.775 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 2 2024-02-13T14:13:37.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:37 smithi055 ceph-mon[25769]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:37.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:37 smithi055 ceph-mon[25769]: Deploying daemon mon.b on smithi078 2024-02-13T14:13:37.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:37 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:38.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:38 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/2041172136' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:38.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:38 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/2041172136' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:39.347 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-13T14:13:39.348 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph mon dump -f json 2024-02-13T14:13:39.596 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:13:39.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:39 smithi170 ceph-mon[30181]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:39.716 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:39 smithi055 ceph-mon[25769]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.081 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:39 smithi078 podman[30048]: 2024-02-13 14:13:39.973450946 +0000 UTC m=+0.120300093 container create 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:13:40.449 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 podman[30048]: 2024-02-13 14:13:40.167252327 +0000 UTC m=+0.314101469 container init 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-13T14:13:40.449 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 podman[30048]: 2024-02-13 14:13:40.172776681 +0000 UTC m=+0.319625825 container start 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:13:40.449 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: set uid:gid to 167:167 (ceph:ceph) 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: ceph version 19.0.0-1423-ge1b79d5a (e1b79d5a4193a484176863ab96c0ca28c00e1662) squid (dev), process ceph-mon, pid 7 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pidfile_write: ignore empty --pid-file 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: load: jerasure load: lrc 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: RocksDB version: 7.9.2 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Git sha 0 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Compile date 2024-02-12 22:40:01 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: DB SUMMARY 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: DB Session ID: TKTHJ3TPRQHTNBVIHZSR 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: CURRENT file: CURRENT 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: IDENTITY file: IDENTITY 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-13T14:13:40.450 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.error_if_exists: 0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.create_if_missing: 0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.paranoid_checks: 1 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.env: 0x557a42b97ae0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.fs: PosixFileSystem 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.info_log: 0x557a447338c0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_file_opening_threads: 16 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.statistics: (nil) 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.use_fsync: 0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_log_file_size: 0 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-13T14:13:40.451 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.keep_log_file_num: 1000 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.recycle_log_file_num: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_fallocate: 1 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_mmap_reads: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_mmap_writes: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.use_direct_reads: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.create_missing_column_families: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.db_log_dir: 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.wal_dir: 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-13T14:13:40.452 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.advise_random_on_open: 1 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.db_write_buffer_size: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.write_buffer_manager: 0x557a44707400 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.rate_limiter: (nil) 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.wal_recovery_mode: 2 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enable_thread_tracking: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enable_pipelined_write: 0 2024-02-13T14:13:40.453 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.unordered_write: 0 2024-02-13T14:13:40.454 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-13T14:13:40.454 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-13T14:13:40.454 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-13T14:13:40.454 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-13T14:13:40.454 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.row_cache: None 2024-02-13T14:13:40.455 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.wal_filter: None 2024-02-13T14:13:40.456 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_ingest_behind: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.two_write_queues: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.manual_wal_flush: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.wal_compression: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.atomic_flush: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.log_readahead_size: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.best_efforts_recovery: 0 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-13T14:13:40.457 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.allow_data_in_errors: 0 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.db_host_id: __hostname__ 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_background_jobs: 2 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_background_compactions: -1 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_subcompactions: 1 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_total_wal_size: 0 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-13T14:13:40.458 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_open_files: -1 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bytes_per_sync: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_readahead_size: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_background_flushes: -1 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Compression algorithms supported: 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kZSTD supported: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kXpressCompression supported: 0 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kLZ4HCCompression supported: 1 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kZlibCompression supported: 1 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kSnappyCompression supported: 1 2024-02-13T14:13:40.459 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kLZ4Compression supported: 1 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: kBZip2Compression supported: 0 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.merge_operator: 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_filter: None 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_filter_factory: None 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.sst_partitioner_factory: None 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-13T14:13:40.460 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557a447339c0) 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: cache_index_and_filter_blocks: 1 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: pin_top_level_index_and_filter: 1 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_type: 0 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: data_block_index_type: 0 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_shortening: 1 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: checksum: 4 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: no_block_cache: 0 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache: 0x557a44737090 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_name: BinnedLRUCache 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_options: 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: capacity : 536870912 2024-02-13T14:13:40.461 INFO:journalctl@ceph.mon.b.smithi078.stdout: num_shard_bits : 4 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: strict_capacity_limit : 0 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: high_pri_pool_ratio: 0.000 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_cache_compressed: (nil) 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: persistent_cache: (nil) 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_size: 4096 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_size_deviation: 10 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_restart_interval: 16 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: index_block_restart_interval: 1 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: metadata_block_size: 4096 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: partition_filters: 0 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: use_delta_encoding: 1 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: filter_policy: bloomfilter 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: whole_key_filtering: 1 2024-02-13T14:13:40.462 INFO:journalctl@ceph.mon.b.smithi078.stdout: verify_compression: 0 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: read_amp_bytes_per_bit: 0 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: format_version: 5 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: enable_index_compression: 1 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: block_align: 0 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: max_auto_readahead_size: 262144 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: prepopulate_block_cache: 0 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: initial_auto_readahead_size: 8192 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout: num_file_reads_for_auto_readahead: 2 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.write_buffer_size: 33554432 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_write_buffer_number: 2 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression: NoCompression 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression: Disabled 2024-02-13T14:13:40.463 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.prefix_extractor: nullptr 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.num_levels: 7 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-13T14:13:40.464 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.level: 32767 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.strategy: 0 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.enabled: false 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-13T14:13:40.465 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.target_file_size_base: 67108864 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-13T14:13:40.466 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.arena_block_size: 1048576 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.disable_auto_compactions: 0 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-13T14:13:40.467 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.table_properties_collectors: 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.inplace_update_support: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.bloom_locality: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.max_successive_merges: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.paranoid_file_checks: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.force_consistency_checks: 1 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.report_bg_io_stats: 0 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.ttl: 2592000 2024-02-13T14:13:40.468 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enable_blob_files: false 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.min_blob_size: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_file_size: 268435456 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.blob_file_starting_level: 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-13T14:13:40.469 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 560e75e7-11d2-4c11-b4c3-b431883c2347 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833620212962, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833620213356, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707833620, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "560e75e7-11d2-4c11-b4c3-b431883c2347", "db_session_id": "TKTHJ3TPRQHTNBVIHZSR", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707833620213432, "job": 1, "event": "recovery_finished"} 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557a44828000 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: DB pointer 0x557a44812000 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** DB Stats ** 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:40.470 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** Compaction Stats [default] ** 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** Compaction Stats [default] ** 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-13T14:13:40.471 INFO:journalctl@ceph.mon.b.smithi078.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.5 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Cumulative compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Interval compaction: 0.00 GB write, 0.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-13T14:13:40.472 INFO:journalctl@ceph.mon.b.smithi078.stdout: Block cache BinnedLRUCache@0x557a44737090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: using public_addr v2:172.21.15.78:0/0 -> [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: starting mon.b rank -1 at public addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] at bind addrs [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(???) e0 preinit fsid 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).mds e1 new map 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: e1 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: legacy client fscid: -1 2024-02-13T14:13:40.473 INFO:journalctl@ceph.mon.b.smithi078.stdout: 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout: No filesystems configured 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Added host smithi078 2024-02-13T14:13:40.474 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi170", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi078", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:40.475 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Deploying cephadm binary to smithi170 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.476 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Added host smithi170 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/1852161889' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.477 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi170", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Saving service mon spec with placement smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c;count:3 2024-02-13T14:13:40.478 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Deploying daemon mon.c on smithi170 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/1843270271' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.a calling monitor election 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.479 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.c calling monitor election 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: monmap e2: 2 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: fsmap 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mgrmap e13: y(active, since 52s) 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: overall HEALTH_OK 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.480 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: Deploying daemon mon.b on smithi078 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/2041172136' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 ceph-mon[30086]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 bash[30048]: 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a 2024-02-13T14:13:40.481 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:40 smithi078 systemd[1]: Started Ceph mon.b for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:13:45.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: mon.a calling monitor election 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: mon.c calling monitor election 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: fsmap 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: mgrmap e13: y(active, since 61s) 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:45.658 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: mon.a calling monitor election 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: mon.c calling monitor election 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: fsmap 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: mgrmap e13: y(active, since 61s) 2024-02-13T14:13:45.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-13T14:13:45.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-13T14:13:45.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-13T14:13:45.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2024-02-13T14:13:45.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:45.817 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:45.937 INFO:teuthology.orchestra.run.smithi170.stdout: 2024-02-13T14:13:45.938 INFO:teuthology.orchestra.run.smithi170.stdout:{"epoch":3,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","modified":"2024-02-13T14:13:40.235817Z","created":"2024-02-13T14:11:40.903138Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:3300","nonce":0},{"type":"v1","addr":"172.21.15.55:6789","nonce":0}]},"addr":"172.21.15.55:6789/0","public_addr":"172.21.15.55:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:3300","nonce":0},{"type":"v1","addr":"172.21.15.170:6789","nonce":0}]},"addr":"172.21.15.170:6789/0","public_addr":"172.21.15.170:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:3300","nonce":0},{"type":"v1","addr":"172.21.15.78:6789","nonce":0}]},"addr":"172.21.15.78:6789/0","public_addr":"172.21.15.78:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-13T14:13:45.940 INFO:teuthology.orchestra.run.smithi170.stderr:dumped monmap epoch 3 2024-02-13T14:13:46.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:46.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:46.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:46.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:46 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/1905596662' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:46.539 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:46.564 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-13T14:13:46.564 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph config generate-minimal-conf 2024-02-13T14:13:46.594 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:46.594 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:46.594 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:46.594 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:46 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/1905596662' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:46.594 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:46.821 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.a calling monitor election 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.c calling monitor election 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:47.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: fsmap 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mgrmap e13: y(active, since 61s) 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.b (rank 2) addr [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] is down (out of quorum) 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:47.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:47.700 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:47.700 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:47.700 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:47.700 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/1905596662' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-13T14:13:47.700 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:48.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mon.b calling monitor election 2024-02-13T14:13:48.051 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mon.b calling monitor election 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mon.c calling monitor election 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mon.a calling monitor election 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: fsmap 2024-02-13T14:13:48.052 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: mgrmap e13: y(active, since 63s) 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: Cluster is now healthy 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: overall HEALTH_OK 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.053 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.054 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:48.054 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:48.054 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:13:48.054 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:48.054 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:47 smithi055 ceph-mon[25769]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:48.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mon.b calling monitor election 2024-02-13T14:13:48.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mon.b calling monitor election 2024-02-13T14:13:48.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mon.c calling monitor election 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mon.a calling monitor election 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: fsmap 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: mgrmap e13: y(active, since 63s) 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: Cluster is now healthy 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: overall HEALTH_OK 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:48.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:13:48.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:48.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:47 smithi170 ceph-mon[30181]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.b calling monitor election 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.b calling monitor election 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.c calling monitor election 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.a calling monitor election 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: monmap e3: 3 mons at {a=[v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0],b=[v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0],c=[v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: fsmap 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: osdmap e4: 0 total, 0 up, 0 in 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: mgrmap e13: y(active, since 63s) 2024-02-13T14:13:48.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Cluster is now healthy 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: overall HEALTH_OK 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:13:48.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:47 smithi078 ceph-mon[30086]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:13:48.842 INFO:teuthology.orchestra.run.smithi055.stdout:# minimal ceph.conf for 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:48.842 INFO:teuthology.orchestra.run.smithi055.stdout:[global] 2024-02-13T14:13:48.842 INFO:teuthology.orchestra.run.smithi055.stdout: fsid = 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:13:48.842 INFO:teuthology.orchestra.run.smithi055.stdout: mon_host = [v2:172.21.15.55:3300/0,v1:172.21.15.55:6789/0] [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] [v2:172.21.15.170:3300/0,v1:172.21.15.170:6789/0] 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: Reconfiguring daemon mon.a on smithi055 2024-02-13T14:13:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.067 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:48 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/202060921' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:49.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:13:49.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: Reconfiguring daemon mon.a on smithi055 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:48 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/202060921' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: Reconfiguring mon.a (unknown last config time)... 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: Reconfiguring daemon mon.a on smithi055 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:48 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/202060921' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:49.498 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-13T14:13:49.498 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:13:49.499 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T14:13:49.569 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:13:49.570 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:49.637 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:13:49.638 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T14:13:49.672 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:13:49.672 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:49.742 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:13:49.742 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.conf 2024-02-13T14:13:49.777 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:13:49.778 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:13:49.847 INFO:tasks.cephadm:Adding mgr.y on smithi055 2024-02-13T14:13:49.848 INFO:tasks.cephadm:Adding mgr.x on smithi078 2024-02-13T14:13:49.848 INFO:tasks.cephadm:Adding mgr.z on smithi170 2024-02-13T14:13:49.848 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply mgr '3;smithi055=y;smithi078=x;smithi170=z' 2024-02-13T14:13:50.122 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:51.626 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:51.627 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:51 smithi170 ceph-mon[30181]: Reconfiguring daemon mon.b on smithi078 2024-02-13T14:13:51.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:51.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:13:51.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:51.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:51.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:51.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:51 smithi078 ceph-mon[30086]: Reconfiguring daemon mon.b on smithi078 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: Reconfiguring mon.b (monmap changed)... 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:51.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:51.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:51 smithi055 ceph-mon[25769]: Reconfiguring daemon mon.b on smithi078 2024-02-13T14:13:51.867 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled mgr update... 2024-02-13T14:13:52.552 DEBUG:teuthology.orchestra.run.smithi078:mgr.x> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.x.service 2024-02-13T14:13:52.555 DEBUG:teuthology.orchestra.run.smithi170:mgr.z> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.z.service 2024-02-13T14:13:52.557 INFO:tasks.cephadm:Deploying OSDs... 2024-02-13T14:13:52.557 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:13:52.557 DEBUG:teuthology.orchestra.run.smithi055:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:13:52.576 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:13:52.577 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/nvme1n1 2024-02-13T14:13:52.634 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/nvme1n1 2024-02-13T14:13:52.634 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:52.634 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:12:56.805004285 +0000 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:10:14.890389503 +0000 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:10:14.890389503 +0000 2024-02-13T14:13:52.635 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:10:14.880389703 +0000 2024-02-13T14:13:52.635 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-13T14:13:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:13:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:13:52.704 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000354285 s, 1.4 MB/s 2024-02-13T14:13:52.705 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-13T14:13:52.764 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/nvme2n1 2024-02-13T14:13:52.821 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/nvme2n1 2024-02-13T14:13:52.821 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:52.821 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-13T14:13:52.821 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:52.822 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:52.822 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:12:56.817004044 +0000 2024-02-13T14:13:52.822 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:10:15.037386564 +0000 2024-02-13T14:13:52.822 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:10:15.037386564 +0000 2024-02-13T14:13:52.822 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:10:15.024386824 +0000 2024-02-13T14:13:52.822 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-13T14:13:52.890 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:13:52.890 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:13:52.890 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000301959 s, 1.7 MB/s 2024-02-13T14:13:52.891 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-13T14:13:52.949 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/nvme3n1 2024-02-13T14:13:53.004 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/nvme3n1 2024-02-13T14:13:53.004 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:12:56.829003804 +0000 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:10:15.189383526 +0000 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:10:15.189383526 +0000 2024-02-13T14:13:53.005 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:10:15.177383766 +0000 2024-02-13T14:13:53.005 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-13T14:13:53.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=y;smithi078=x;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: Saving service mgr spec with placement smithi055=y;smithi078=x;smithi170=z;count:3 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:52 smithi055 ceph-mon[25769]: Reconfiguring daemon mon.c on smithi170 2024-02-13T14:13:53.071 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:13:53.071 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:13:53.071 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000292585 s, 1.7 MB/s 2024-02-13T14:13:53.072 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-13T14:13:53.129 DEBUG:teuthology.orchestra.run.smithi055:> stat /dev/nvme4n1 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=y;smithi078=x;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: Saving service mgr spec with placement smithi055=y;smithi078=x;smithi170=z;count:3 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:52 smithi170 ceph-mon[30181]: Reconfiguring daemon mon.c on smithi170 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout: File: /dev/nvme4n1 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Device: 5h/5d Inode: 798 Links: 1 Device type: 103,8 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Access: 2024-02-13 14:12:56.844003504 +0000 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Modify: 2024-02-13 14:10:15.331380687 +0000 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout:Change: 2024-02-13 14:10:15.331380687 +0000 2024-02-13T14:13:53.186 INFO:teuthology.orchestra.run.smithi055.stdout: Birth: 2024-02-13 14:10:15.324380827 +0000 2024-02-13T14:13:53.187 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi055=y;smithi078=x;smithi170=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: Saving service mgr spec with placement smithi055=y;smithi078=x;smithi170=z;count:3 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: Reconfiguring mon.c (monmap changed)... 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:52 smithi078 ceph-mon[30086]: Reconfiguring daemon mon.c on smithi170 2024-02-13T14:13:53.253 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records in 2024-02-13T14:13:53.253 INFO:teuthology.orchestra.run.smithi055.stderr:1+0 records out 2024-02-13T14:13:53.253 INFO:teuthology.orchestra.run.smithi055.stderr:512 bytes copied, 0.000352899 s, 1.5 MB/s 2024-02-13T14:13:53.254 DEBUG:teuthology.orchestra.run.smithi055:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-13T14:13:53.310 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:13:53.311 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:13:53.329 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:13:53.329 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme1n1 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme1n1 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:13:12.868277905 +0000 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:10:16.542167334 +0000 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:10:16.542167334 +0000 2024-02-13T14:13:53.387 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:10:16.530168618 +0000 2024-02-13T14:13:53.388 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-13T14:13:53.456 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:13:53.456 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:13:53.456 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000287717 s, 1.8 MB/s 2024-02-13T14:13:53.457 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-13T14:13:53.515 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme2n1 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme2n1 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:13:12.878277740 +0000 2024-02-13T14:13:53.572 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:10:16.698156198 +0000 2024-02-13T14:13:53.573 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:10:16.698156198 +0000 2024-02-13T14:13:53.573 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:10:16.686156392 +0000 2024-02-13T14:13:53.573 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-13T14:13:53.641 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:13:53.642 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:13:53.642 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000239158 s, 2.1 MB/s 2024-02-13T14:13:53.643 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-13T14:13:53.699 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme3n1 2024-02-13T14:13:53.754 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme3n1 2024-02-13T14:13:53.754 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.754 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 786 Links: 1 Device type: 103,6 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:13:12.889277558 +0000 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:10:16.848153772 +0000 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:10:16.848153772 +0000 2024-02-13T14:13:53.755 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:10:16.831154047 +0000 2024-02-13T14:13:53.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-13T14:13:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:13:53.820 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:13:53.821 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000239327 s, 2.1 MB/s 2024-02-13T14:13:53.822 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-13T14:13:53.879 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/nvme4n1 2024-02-13T14:13:53.934 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/nvme4n1 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-02-13 14:13:12.900277376 +0000 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-02-13 14:10:16.994151410 +0000 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-02-13 14:10:16.994151410 +0000 2024-02-13T14:13:53.935 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: 2024-02-13 14:10:16.986151539 +0000 2024-02-13T14:13:53.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-13T14:13:54.000 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-02-13T14:13:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-02-13T14:13:54.001 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000248107 s, 2.1 MB/s 2024-02-13T14:13:54.002 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-13T14:13:54.058 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:13:54.058 DEBUG:teuthology.orchestra.run.smithi170:> dd if=/scratch_devs of=/dev/stdout 2024-02-13T14:13:54.075 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-13T14:13:54.075 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/nvme1n1 2024-02-13T14:13:54.131 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/nvme1n1 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:13:22.746061988 +0000 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:10:18.226246859 +0000 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:10:18.226246859 +0000 2024-02-13T14:13:54.132 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:10:18.216247086 +0000 2024-02-13T14:13:54.132 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-13T14:13:54.199 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:13:54.199 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:13:54.199 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000140575 s, 3.6 MB/s 2024-02-13T14:13:54.200 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-13T14:13:54.257 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/nvme2n1 2024-02-13T14:13:54.312 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/nvme2n1 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:13:22.754061806 +0000 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:10:18.379243392 +0000 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:10:18.379243392 +0000 2024-02-13T14:13:54.313 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:10:18.369243619 +0000 2024-02-13T14:13:54.313 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-13T14:13:54.378 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:13:54.378 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:13:54.378 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000129757 s, 3.9 MB/s 2024-02-13T14:13:54.379 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-13T14:13:54.435 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/nvme3n1 2024-02-13T14:13:54.490 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/nvme3n1 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:13:22.763061602 +0000 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:10:18.485240991 +0000 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:10:18.485240991 +0000 2024-02-13T14:13:54.491 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:10:18.477241172 +0000 2024-02-13T14:13:54.491 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-13T14:13:54.557 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:13:54.559 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:13:54.559 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.000107425 s, 4.8 MB/s 2024-02-13T14:13:54.560 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-13T14:13:54.615 DEBUG:teuthology.orchestra.run.smithi170:> stat /dev/nvme4n1 2024-02-13T14:13:54.670 INFO:teuthology.orchestra.run.smithi170.stdout: File: /dev/nvme4n1 2024-02-13T14:13:54.670 INFO:teuthology.orchestra.run.smithi170.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-13T14:13:54.670 INFO:teuthology.orchestra.run.smithi170.stdout:Device: 5h/5d Inode: 800 Links: 1 Device type: 103,8 2024-02-13T14:13:54.670 INFO:teuthology.orchestra.run.smithi170.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-13T14:13:54.670 INFO:teuthology.orchestra.run.smithi170.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-13T14:13:54.671 INFO:teuthology.orchestra.run.smithi170.stdout:Access: 2024-02-13 14:13:22.771061420 +0000 2024-02-13T14:13:54.671 INFO:teuthology.orchestra.run.smithi170.stdout:Modify: 2024-02-13 14:10:18.635237591 +0000 2024-02-13T14:13:54.671 INFO:teuthology.orchestra.run.smithi170.stdout:Change: 2024-02-13 14:10:18.635237591 +0000 2024-02-13T14:13:54.671 INFO:teuthology.orchestra.run.smithi170.stdout: Birth: 2024-02-13 14:10:18.632237659 +0000 2024-02-13T14:13:54.671 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-13T14:13:54.739 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records in 2024-02-13T14:13:54.739 INFO:teuthology.orchestra.run.smithi170.stderr:1+0 records out 2024-02-13T14:13:54.739 INFO:teuthology.orchestra.run.smithi170.stderr:512 bytes copied, 0.00010487 s, 4.9 MB/s 2024-02-13T14:13:54.740 DEBUG:teuthology.orchestra.run.smithi170:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-13T14:13:54.799 INFO:tasks.cephadm:Deploying osd.0 on smithi055 with /dev/nvme4n1... 2024-02-13T14:13:54.799 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:54.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:54.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:54 smithi170 ceph-mon[30181]: Deploying daemon mgr.z on smithi170 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:54.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:54.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:54.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:54.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:54 smithi078 ceph-mon[30086]: Deploying daemon mgr.z on smithi170 2024-02-13T14:13:55.036 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:55.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:55.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:54 smithi055 ceph-mon[25769]: Deploying daemon mgr.z on smithi170 2024-02-13T14:13:56.633 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 podman[32254]: 2024-02-13 14:13:56.469545576 +0000 UTC m=+0.018122732 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:13:56.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:56 smithi078 ceph-mon[30086]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:57.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:56 smithi055 ceph-mon[25769]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping: /dev/nvme4n1 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10+0 records in 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr:10+0 records out 2024-02-13T14:13:57.120 INFO:teuthology.orchestra.run.smithi055.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00997432 s, 1.1 GB/s 2024-02-13T14:13:57.121 INFO:teuthology.orchestra.run.smithi055.stderr:--> Zapping successful for: 2024-02-13T14:13:57.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:56 smithi170 ceph-mon[30181]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:57.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 podman[32254]: 2024-02-13 14:13:56.728379316 +0000 UTC m=+0.276956466 container init a56913adfd5103788942e0f9c5618aac491d15495bf367c38a911b71921b97d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-13T14:13:57.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 podman[32254]: 2024-02-13 14:13:56.733562909 +0000 UTC m=+0.282140051 container start a56913adfd5103788942e0f9c5618aac491d15495bf367c38a911b71921b97d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux ) 2024-02-13T14:13:57.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 ceph-mgr[32292]: -- 172.21.15.170:0/3117168334 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562eae8bc1a0 con 0x562eae893000 2024-02-13T14:13:57.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 bash[32254]: a56913adfd5103788942e0f9c5618aac491d15495bf367c38a911b71921b97d0 2024-02-13T14:13:57.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:56 smithi170 systemd[1]: Started Ceph mgr.z for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:13:57.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:57.269+0000 7f6f8248f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:13:57.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:57.360+0000 7f6f8248f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:13:57.771 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi055:/dev/nvme4n1 2024-02-13T14:13:58.014 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:58.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:57 smithi055 ceph-mon[25769]: Deploying daemon mgr.x on smithi078 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:58.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:58.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:58.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-mon[30181]: Deploying daemon mgr.x on smithi078 2024-02-13T14:13:58.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:57 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:57.995+0000 7f6f8248f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:13:58.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.088+0000 7f6f8248f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:13:58.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:57 smithi078 ceph-mon[30086]: Deploying daemon mgr.x on smithi078 2024-02-13T14:13:58.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.237+0000 7f6f8248f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:13:58.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.331+0000 7f6f8248f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:13:58.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.413+0000 7f6f8248f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:13:58.958 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.673+0000 7f6f8248f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:13:58.958 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.780+0000 7f6f8248f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:13:58.958 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.864+0000 7f6f8248f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:13:58.958 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:58.956+0000 7f6f8248f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:13:59.056 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:13:58 smithi055 ceph-mon[25769]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:59.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:13:58 smithi078 ceph-mon[30086]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:59.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 podman[31525]: 2024-02-13 14:13:59.126202409 +0000 UTC m=+0.117597046 container create cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-13T14:13:59.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:13:58 smithi170 ceph-mon[30181]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:13:59.406 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:13:59 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:13:59.131+0000 7f6f8248f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:13:59.642 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 podman[31525]: 2024-02-13 14:13:59.345036091 +0000 UTC m=+0.336430683 container init cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-13T14:13:59.642 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 podman[31525]: 2024-02-13 14:13:59.350916856 +0000 UTC m=+0.342311448 container start cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:13:59.642 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 ceph-mgr[31564]: -- 172.21.15.78:0/148181288 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55883b26e1a0 con 0x55883b244800 2024-02-13T14:13:59.642 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 bash[31525]: cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 2024-02-13T14:13:59.642 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 systemd[1]: Started Ceph mgr.x for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:14:00.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:13:59 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:13:59.943+0000 7fc3fc6be200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:14:00.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:00.040+0000 7fc3fc6be200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:14:00.406 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:00.119+0000 7f6f8248f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:14:00.406 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:00.264+0000 7f6f8248f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:00.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:00.756 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:00.619+0000 7f6f8248f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:14:00.756 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:00.715+0000 7f6f8248f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:14:00.756 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:00.757 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:00.762 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:00.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:00.763 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:00.764 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:00.764 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:00 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:00.973 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:00.738+0000 7fc3fc6be200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:14:00.974 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:00.831+0000 7fc3fc6be200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:14:01.102 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:00 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:00.909+0000 7f6f8248f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:14:01.381 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:01.099+0000 7f6f8248f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:14:01.423 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:00 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:00.971+0000 7fc3fc6be200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:14:01.423 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.075+0000 7fc3fc6be200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:14:01.423 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.160+0000 7fc3fc6be200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:14:01.423 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.421+0000 7fc3fc6be200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:14:01.641 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:01.379+0000 7f6f8248f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:14:01.641 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:14:01.642 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.529+0000 7fc3fc6be200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.614+0000 7fc3fc6be200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:01.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:14:01.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:14:01.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.779 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi055:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-13T14:14:01.780 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:01 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:01.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:01.640+0000 7f6f8248f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:14:01.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:01 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:01.876+0000 7f6f8248f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:14:02.197 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.709+0000 7fc3fc6be200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:14:02.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:01 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:01.883+0000 7fc3fc6be200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:14:02.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:02 smithi055 ceph-mon[25769]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:14:02.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:02 smithi055 ceph-mon[25769]: Reconfiguring daemon mgr.y on smithi055 2024-02-13T14:14:02.855 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:02 smithi078 ceph-mon[30086]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:14:02.855 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:02 smithi078 ceph-mon[30086]: Reconfiguring daemon mgr.y on smithi055 2024-02-13T14:14:02.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:02 smithi170 ceph-mon[30181]: Reconfiguring mgr.y (unknown last config time)... 2024-02-13T14:14:02.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:02 smithi170 ceph-mon[30181]: Reconfiguring daemon mgr.y on smithi055 2024-02-13T14:14:02.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:02 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:02.676+0000 7f6f8248f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:14:03.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:02 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:02.853+0000 7fc3fc6be200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:14:03.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:02 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:02.981+0000 7fc3fc6be200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:14:03.549 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:03.278+0000 7f6f8248f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:14:03.549 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:03.374+0000 7f6f8248f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:14:03.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-mon[30086]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:03.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]: dispatch 2024-02-13T14:14:03.583 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]': finished 2024-02-13T14:14:03.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-mon[30086]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:14:03.584 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:03.584 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:03.305+0000 7fc3fc6be200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:14:03.584 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:03.391+0000 7fc3fc6be200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:14:03.584 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:03.581+0000 7fc3fc6be200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:14:03.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:03 smithi055 ceph-mon[25769]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:03.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:03 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]: dispatch 2024-02-13T14:14:03.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:03 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]': finished 2024-02-13T14:14:03.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:03 smithi055 ceph-mon[25769]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:14:03.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:03 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:03.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:14:03.590+0000 7f6f8248f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:14:03.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-mon[30181]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:03.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]: dispatch 2024-02-13T14:14:03.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2321202285' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3667302d-4268-42ea-9250-c1a648747a0b"}]': finished 2024-02-13T14:14:03.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-mon[30181]: osdmap e5: 1 total, 0 up, 1 in 2024-02-13T14:14:03.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:03 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:03.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:03 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:03.763+0000 7fc3fc6be200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:14:04.304 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:04.044+0000 7fc3fc6be200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:14:04.304 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:04.302+0000 7fc3fc6be200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:04.536+0000 7fc3fc6be200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: Standby manager daemon z started 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:04 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/4140509499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:04.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: Standby manager daemon z started 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:04.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:04 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4140509499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: Standby manager daemon z started 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/603359337' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:04.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:04 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/4140509499' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:05 smithi078 ceph-mon[30086]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:05 smithi078 ceph-mon[30086]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-13T14:14:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:05 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:14:05.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:05 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:05.332+0000 7fc3fc6be200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:14:05.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:05 smithi055 ceph-mon[25769]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:05.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:05 smithi055 ceph-mon[25769]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-13T14:14:05.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:05 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:14:05.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:05 smithi170 ceph-mon[30181]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:05.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:05 smithi170 ceph-mon[30181]: mgrmap e14: y(active, since 80s), standbys: z 2024-02-13T14:14:05.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:05 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:14:06.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:05 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:05.903+0000 7fc3fc6be200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:14:06.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:06.000+0000 7fc3fc6be200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:14:06.559 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:14:06.214+0000 7fc3fc6be200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:14:06.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:06 smithi055 ceph-mon[25769]: Standby manager daemon x started 2024-02-13T14:14:06.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:06 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:14:06.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:06 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:06.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:06 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:14:06.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:06 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:06.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:06 smithi170 ceph-mon[30181]: Standby manager daemon x started 2024-02-13T14:14:06.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:06 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:14:06.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:06 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:06.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:06 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:14:06.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:06 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:06.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-mon[30086]: Standby manager daemon x started 2024-02-13T14:14:06.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:14:06.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:14:06.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:14:06.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:06 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/2377070458' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:14:07.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:07 smithi055 ceph-mon[25769]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:07.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:07 smithi055 ceph-mon[25769]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-13T14:14:07.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:14:07.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:07 smithi170 ceph-mon[30181]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:07.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:07 smithi170 ceph-mon[30181]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-13T14:14:07.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:14:07.947 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:07 smithi078 ceph-mon[30086]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:07.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:07 smithi078 ceph-mon[30086]: mgrmap e15: y(active, since 82s), standbys: z, x 2024-02-13T14:14:07.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:09.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:11.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:11 smithi078 ceph-mon[30086]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:11.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:11 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:14:11.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:11 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:11.496 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:11 smithi055 ceph-mon[25769]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:11.496 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:11 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:14:11.496 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:11 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:11 smithi170 ceph-mon[30181]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:11 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-13T14:14:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:11 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:12.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:12 smithi078 ceph-mon[30086]: Deploying daemon osd.0 on smithi055 2024-02-13T14:14:12.462 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:12 smithi055 ceph-mon[25769]: Deploying daemon osd.0 on smithi055 2024-02-13T14:14:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:12 smithi170 ceph-mon[30181]: Deploying daemon osd.0 on smithi055 2024-02-13T14:14:13.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:13 smithi055 ceph-mon[25769]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:13 smithi170 ceph-mon[30181]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:13 smithi078 ceph-mon[30086]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:15.542 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:15 smithi055 ceph-mon[25769]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:15.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:15 smithi170 ceph-mon[30181]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:15.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:15 smithi078 ceph-mon[30086]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:17 smithi055 ceph-mon[25769]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:17.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:17 smithi170 ceph-mon[30181]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:17.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:17 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:17 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:17 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:17 smithi078 ceph-mon[30086]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:17 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:17 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:17 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:19.556 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:19 smithi055 ceph-mon[25769]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:19.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:19 smithi170 ceph-mon[30181]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:19.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:19 smithi078 ceph-mon[30086]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:20.495 INFO:teuthology.orchestra.run.smithi055.stdout:Created osd(s) 0 on host 'smithi055' 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.802 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:14:20.803 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.803 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:20.803 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:21.124 DEBUG:teuthology.orchestra.run.smithi055:osd.0> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.0.service 2024-02-13T14:14:21.126 INFO:tasks.cephadm:Deploying osd.1 on smithi078 with /dev/nvme4n1... 2024-02-13T14:14:21.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-13T14:14:21.368 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:21.775 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:21 smithi078 ceph-mon[30086]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:21.776 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:21 smithi078 ceph-mon[30086]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:14:21.776 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:21 smithi078 ceph-mon[30086]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:14:21.776 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:21 smithi078 ceph-mon[30086]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-02-13T14:14:21.776 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:21 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:21 smithi055 ceph-mon[25769]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:21 smithi055 ceph-mon[25769]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:14:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:21 smithi055 ceph-mon[25769]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:14:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:21 smithi055 ceph-mon[25769]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-02-13T14:14:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:21 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:21 smithi170 ceph-mon[30181]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:21 smithi170 ceph-mon[30181]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-13T14:14:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:21 smithi170 ceph-mon[30181]: osdmap e6: 1 total, 0 up, 1 in 2024-02-13T14:14:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:21 smithi170 ceph-mon[30181]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]: dispatch 2024-02-13T14:14:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:21 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.519 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0[44231]: 2024-02-13T14:14:22.114+0000 7f901ea46700 -1 osd.0 0 waiting for initial osdmap 2024-02-13T14:14:22.520 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0[44231]: 2024-02-13T14:14:22.120+0000 7f9016831700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' 2024-02-13T14:14:22.789 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' 2024-02-13T14:14:22.823 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi055", "root=default"]}]': finished 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: osdmap e7: 1 total, 0 up, 1 in 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: from='osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115]' entity='osd.0' 2024-02-13T14:14:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:23.412 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping: /dev/nvme4n1 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10+0 records in 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr:10+0 records out 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125575 s, 835 MB/s 2024-02-13T14:14:23.413 INFO:teuthology.orchestra.run.smithi078.stderr:--> Zapping successful for: 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: purged_snaps scrub starts 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: purged_snaps scrub ok 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115] boot 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:14:23.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:23 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:23.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: purged_snaps scrub starts 2024-02-13T14:14:23.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: purged_snaps scrub ok 2024-02-13T14:14:23.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:23.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115] boot 2024-02-13T14:14:23.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:14:23.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:23 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: purged_snaps scrub starts 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: purged_snaps scrub ok 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: osd.0 [v2:172.21.15.55:6802/2907918115,v1:172.21.15.55:6803/2907918115] boot 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: osdmap e8: 1 total, 1 up, 1 in 2024-02-13T14:14:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:23 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:14:23.936 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi078:/dev/nvme4n1 2024-02-13T14:14:24.169 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:14:25.642 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: Detected new or changed devices on smithi055 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: Adjusting osd_memory_target on smithi055 to 17093M 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.643 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:25 smithi078 ceph-mon[30086]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: Detected new or changed devices on smithi055 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: Adjusting osd_memory_target on smithi055 to 17093M 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:25.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:25 smithi055 ceph-mon[25769]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: Detected new or changed devices on smithi055 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: Adjusting osd_memory_target on smithi055 to 17093M 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:25.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:25.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:25.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:25 smithi170 ceph-mon[30181]: osdmap e9: 1 total, 1 up, 1 in 2024-02-13T14:14:26.812 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:26 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:26.812 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:26 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:26.812 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:26 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:26.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:26 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:26.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:26 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:26.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:26 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:26.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:26 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:26.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:26 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:26.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:26 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:27.751 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:27 smithi078 ceph-mon[30086]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:27.751 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:27 smithi078 ceph-mon[30086]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:27.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:27 smithi055 ceph-mon[25769]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:27.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:27 smithi055 ceph-mon[25769]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:27.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:27 smithi170 ceph-mon[30181]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:27.852 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:27 smithi170 ceph-mon[30181]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi078:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:28.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:28 smithi055 ceph-mon[25769]: from='client.? 172.21.15.78:0/1573581146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:28 smithi055 ceph-mon[25769]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:28 smithi055 ceph-mon[25769]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]': finished 2024-02-13T14:14:28.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:28 smithi055 ceph-mon[25769]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:14:28.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:28 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:28.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:28 smithi170 ceph-mon[30181]: from='client.? 172.21.15.78:0/1573581146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:28 smithi170 ceph-mon[30181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:28 smithi170 ceph-mon[30181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]': finished 2024-02-13T14:14:28.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:28 smithi170 ceph-mon[30181]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:14:28.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:28 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:28.909 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:28 smithi078 ceph-mon[30086]: from='client.? 172.21.15.78:0/1573581146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.909 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:28 smithi078 ceph-mon[30086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]: dispatch 2024-02-13T14:14:28.909 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:28 smithi078 ceph-mon[30086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "729b1700-978e-4a40-a405-13b420956686"}]': finished 2024-02-13T14:14:28.910 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:28 smithi078 ceph-mon[30086]: osdmap e10: 2 total, 1 up, 2 in 2024-02-13T14:14:28.910 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:28 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:29.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:29 smithi055 ceph-mon[25769]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:29.827 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:29 smithi078 ceph-mon[30086]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:29 smithi170 ceph-mon[30181]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:30.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:30 smithi055 ceph-mon[25769]: from='client.? 172.21.15.78:0/4024472262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:30 smithi170 ceph-mon[30181]: from='client.? 172.21.15.78:0/4024472262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:30 smithi078 ceph-mon[30086]: from='client.? 172.21.15.78:0/4024472262' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:31.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:31 smithi055 ceph-mon[25769]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:31.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:31 smithi170 ceph-mon[30181]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:31.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:31 smithi078 ceph-mon[30086]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:33.806 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:33 smithi078 ceph-mon[30086]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:33.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:33 smithi055 ceph-mon[25769]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:33.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:33 smithi170 ceph-mon[30181]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:35.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:35 smithi078 ceph-mon[30086]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:35.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:35 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:14:35.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:35 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:35 smithi055 ceph-mon[25769]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:35.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:35 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:14:35.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:35 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:35.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:35 smithi170 ceph-mon[30181]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:35.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:35 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-13T14:14:35.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:35 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:36.586 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:36 smithi078 ceph-mon[30086]: Deploying daemon osd.1 on smithi078 2024-02-13T14:14:36.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:36 smithi055 ceph-mon[25769]: Deploying daemon osd.1 on smithi078 2024-02-13T14:14:36.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:36 smithi170 ceph-mon[30181]: Deploying daemon osd.1 on smithi078 2024-02-13T14:14:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:37 smithi170 ceph-mon[30181]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:37.851 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:37 smithi078 ceph-mon[30086]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:38.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:37 smithi055 ceph-mon[25769]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:39.833 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:39 smithi078 ceph-mon[30086]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:39.867 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:39 smithi055 ceph-mon[25769]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:39.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:39 smithi170 ceph-mon[30181]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:41.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:41 smithi078 ceph-mon[30086]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:41.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:41 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:41.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:41 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:41.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:41 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:41.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:41 smithi170 ceph-mon[30181]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:41.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:41 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:41.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:41 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:41.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:41 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:42.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:41 smithi055 ceph-mon[25769]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:42.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:41 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:42.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:41 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:42.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:41 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:43.804 INFO:teuthology.orchestra.run.smithi078.stdout:Created osd(s) 1 on host 'smithi078' 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:43.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:44.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.438 DEBUG:teuthology.orchestra.run.smithi078:osd.1> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.1.service 2024-02-13T14:14:44.440 INFO:tasks.cephadm:Deploying osd.2 on smithi170 with /dev/nvme4n1... 2024-02-13T14:14:44.440 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-13T14:14:44.685 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:14:44.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:44 smithi078 ceph-mon[30086]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:44.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:44 smithi078 ceph-mon[30086]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:44.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:44 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:44 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:44 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:44.807 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:44 smithi170 ceph-mon[30181]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:44.807 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:44 smithi170 ceph-mon[30181]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:44.807 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:44 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.807 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:44 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:44.807 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:44 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:44 smithi055 ceph-mon[25769]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:44 smithi055 ceph-mon[25769]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-13T14:14:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:45.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:45.661 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:45 smithi170 ceph-mon[30181]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:45.814 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:45 smithi078 ceph-mon[30086]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:46.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:46.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-13T14:14:46.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: from='osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:46.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: osdmap e11: 2 total, 1 up, 2 in 2024-02-13T14:14:46.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:46.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:45 smithi055 ceph-mon[25769]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]: dispatch 2024-02-13T14:14:46.448 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1[34638]: 2024-02-13T14:14:46.097+0000 7fe0fb991700 -1 osd.1 0 waiting for initial osdmap 2024-02-13T14:14:46.448 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1[34638]: 2024-02-13T14:14:46.099+0000 7fe0f377c700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping: /dev/nvme4n1 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr: stderr: 10+0 records in 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:10+0 records out 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00942239 s, 1.1 GB/s 2024-02-13T14:14:46.510 INFO:teuthology.orchestra.run.smithi170.stderr:--> Zapping successful for: 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:46.834 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:46 smithi170 ceph-mon[30181]: from='osd.1 ' entity='osd.1' 2024-02-13T14:14:46.947 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-13T14:14:46.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:14:46.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:46.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:46.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:46.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:46 smithi078 ceph-mon[30086]: from='osd.1 ' entity='osd.1' 2024-02-13T14:14:47.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi078", "root=default"]}]': finished 2024-02-13T14:14:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: osdmap e12: 2 total, 1 up, 2 in 2024-02-13T14:14:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: pgmap v63: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-02-13T14:14:47.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:46 smithi055 ceph-mon[25769]: from='osd.1 ' entity='osd.1' 2024-02-13T14:14:47.095 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch daemon add osd smithi170:/dev/nvme4n1 2024-02-13T14:14:47.330 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:14:47.711 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: purged_snaps scrub starts 2024-02-13T14:14:47.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: purged_snaps scrub ok 2024-02-13T14:14:47.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:47.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124] boot 2024-02-13T14:14:47.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:14:47.712 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:47 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: purged_snaps scrub starts 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: purged_snaps scrub ok 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124] boot 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:14:47.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:47 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: purged_snaps scrub starts 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: purged_snaps scrub ok 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: osd.1 [v2:172.21.15.78:6800/3405515124,v1:172.21.15.78:6801/3405515124] boot 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: osdmap e13: 2 total, 2 up, 2 in 2024-02-13T14:14:48.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:47 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:14:48.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:48.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: Detected new or changed devices on smithi078 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:48.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:48 smithi170 ceph-mon[30181]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: Detected new or changed devices on smithi078 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:48 smithi055 ceph-mon[25769]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: Detected new or changed devices on smithi078 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:14:49.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: Adjusting osd_memory_target on smithi078 to 17088M 2024-02-13T14:14:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:14:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:14:49.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:48 smithi078 ceph-mon[30086]: osdmap e14: 2 total, 2 up, 2 in 2024-02-13T14:14:49.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:49 smithi170 ceph-mon[30181]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:49.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:49 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:49.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:49 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:49.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:49 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:50.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:49 smithi055 ceph-mon[25769]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:50.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:49 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:50.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:49 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:50.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:49 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:50.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:49 smithi078 ceph-mon[30086]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi170:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:14:50.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:49 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-13T14:14:50.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:49 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-13T14:14:50.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:49 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:14:50.971 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:50 smithi170 ceph-mon[30181]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:51.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:50 smithi055 ceph-mon[25769]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:51.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:50 smithi078 ceph-mon[30086]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/741655620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]': finished 2024-02-13T14:14:53.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:14:53.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:52 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:14:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:53.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/741655620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]': finished 2024-02-13T14:14:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:14:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:52 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/741655620' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]: dispatch 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1cd8c691-2f2e-4edf-a116-b96a87f2c815"}]': finished 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: osdmap e15: 3 total, 2 up, 3 in 2024-02-13T14:14:53.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:52 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:14:54.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:53 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/443271073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:53 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/443271073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:53 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/443271073' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-13T14:14:55.057 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:54 smithi170 ceph-mon[30181]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:55.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:54 smithi055 ceph-mon[25769]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:54 smithi078 ceph-mon[30086]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:57.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:56 smithi055 ceph-mon[25769]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:57.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:56 smithi170 ceph-mon[30181]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:57.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:56 smithi078 ceph-mon[30086]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:59.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:58 smithi055 ceph-mon[25769]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:59.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:58 smithi170 ceph-mon[30181]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:14:59.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:58 smithi078 ceph-mon[30086]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:00.077 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:59 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:15:00.078 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:59 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:00.078 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:14:59 smithi170 ceph-mon[30181]: Deploying daemon osd.2 on smithi170 2024-02-13T14:15:00.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:59 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:15:00.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:59 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:00.151 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:14:59 smithi078 ceph-mon[30086]: Deploying daemon osd.2 on smithi170 2024-02-13T14:15:00.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-13T14:15:00.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:59 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:00.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:14:59 smithi055 ceph-mon[25769]: Deploying daemon osd.2 on smithi170 2024-02-13T14:15:00.998 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:00 smithi170 ceph-mon[30181]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:01.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:00 smithi078 ceph-mon[30086]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:01.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:00 smithi055 ceph-mon[25769]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:02.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:02 smithi170 ceph-mon[30181]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:03.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:02 smithi078 ceph-mon[30086]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:03.314 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:02 smithi055 ceph-mon[25769]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:05.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:05 smithi170 ceph-mon[30181]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:05.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:05 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:05 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:05 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:05.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:05 smithi055 ceph-mon[25769]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:05.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:05 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:05 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:05 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:05 smithi078 ceph-mon[30086]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:05 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:05 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:05.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:05 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:07.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:07 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.610 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:07.610 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.610 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.610 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:07.611 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:07.611 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:07 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:07.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:07 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:07.986 INFO:teuthology.orchestra.run.smithi170.stdout:Created osd(s) 2 on host 'smithi170' 2024-02-13T14:15:08.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:08 smithi055 ceph-mon[25769]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:08 smithi055 ceph-mon[25769]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:08 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.572 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:08 smithi170 ceph-mon[30181]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.572 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:08 smithi170 ceph-mon[30181]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.572 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:08 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.572 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:08 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.572 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:08 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.601 DEBUG:teuthology.orchestra.run.smithi170:osd.2> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.2.service 2024-02-13T14:15:08.603 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-13T14:15:08.603 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd stat -f json 2024-02-13T14:15:08.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:08 smithi078 ceph-mon[30086]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:08 smithi078 ceph-mon[30086]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-13T14:15:08.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:08 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:08.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:08 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:08 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:08.856 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.529 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-mon[30181]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.564 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:09.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:15:09.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:15:09.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:09.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:09 smithi055 ceph-mon[25769]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.655 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:09.655 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-13T14:15:09.655 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: osdmap e16: 3 total, 2 up, 3 in 2024-02-13T14:15:09.655 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:09.655 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: from='osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.656 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:09 smithi078 ceph-mon[30086]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]: dispatch 2024-02-13T14:15:09.817 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2[35448]: 2024-02-13T14:15:09.527+0000 7f8fd10fb700 -1 osd.2 0 waiting for initial osdmap 2024-02-13T14:15:09.817 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:09 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2[35448]: 2024-02-13T14:15:09.530+0000 7f8fc8ee6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-13T14:15:10.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:10 smithi170 ceph-mon[30181]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-02-13T14:15:10.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:10 smithi170 ceph-mon[30181]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:15:10.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:10 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:10 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:10 smithi170 ceph-mon[30181]: from='osd.2 ' entity='osd.2' 2024-02-13T14:15:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:10 smithi055 ceph-mon[25769]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-02-13T14:15:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:10 smithi055 ceph-mon[25769]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:15:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:10 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:10 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:10 smithi055 ceph-mon[25769]: from='osd.2 ' entity='osd.2' 2024-02-13T14:15:10.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:10 smithi078 ceph-mon[30086]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi170", "root=default"]}]': finished 2024-02-13T14:15:10.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:10 smithi078 ceph-mon[30086]: osdmap e17: 3 total, 2 up, 3 in 2024-02-13T14:15:10.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:10 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:10 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:10.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:10 smithi078 ceph-mon[30086]: from='osd.2 ' entity='osd.2' 2024-02-13T14:15:10.705 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:15:11.449 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: purged_snaps scrub starts 2024-02-13T14:15:11.449 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: purged_snaps scrub ok 2024-02-13T14:15:11.449 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:11.449 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.450 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618] boot 2024-02-13T14:15:11.450 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:15:11.450 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.450 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:11 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3329902998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:15:11.476 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":18,"num_osds":3,"num_up_osds":3,"osd_up_since":1707833710,"num_in_osds":3,"osd_in_since":1707833691,"num_remapped_pgs":0} 2024-02-13T14:15:11.477 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: purged_snaps scrub starts 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: purged_snaps scrub ok 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618] boot 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:11 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3329902998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: purged_snaps scrub starts 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: purged_snaps scrub ok 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: osd.2 [v2:172.21.15.170:6800/2448510618,v1:172.21.15.170:6801/2448510618] boot 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: osdmap e18: 3 total, 3 up, 3 in 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:11.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:11 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3329902998' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-13T14:15:11.719 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:12.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:12 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:12.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:12.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:12 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:12 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: Detected new or changed devices on smithi170 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: Adjusting osd_memory_target on smithi170 to 17088M 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:15:13.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:13 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:13.608 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:15:13.608 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":20,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:11:43.841892+0000","modified":"2024-02-13T14:15:13.259036+0000","last_up_change":"2024-02-13T14:15:10.530332+0000","last_in_change":"2024-02-13T14:14:51.816199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:15:11.960022+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3667302d-4268-42ea-9250-c1a648747a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6803","nonce":2907918115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6805","nonce":2907918115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6809","nonce":2907918115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6807","nonce":2907918115}]},"public_addr":"172.21.15.55:6803/2907918115","cluster_addr":"172.21.15.55:6805/2907918115","heartbeat_back_addr":"172.21.15.55:6809/2907918115","heartbeat_front_addr":"172.21.15.55:6807/2907918115","state":["exists","up"]},{"osd":1,"uuid":"729b1700-978e-4a40-a405-13b420956686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6801","nonce":3405515124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6803","nonce":3405515124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6807","nonce":3405515124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6805","nonce":3405515124}]},"public_addr":"172.21.15.78:6801/3405515124","cluster_addr":"172.21.15.78:6803/3405515124","heartbeat_back_addr":"172.21.15.78:6807/3405515124","heartbeat_front_addr":"172.21.15.78:6805/3405515124","state":["exists","up"]},{"osd":2,"uuid":"1cd8c691-2f2e-4edf-a116-b96a87f2c815","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6801","nonce":2448510618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6803","nonce":2448510618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6807","nonce":2448510618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6805","nonce":2448510618}]},"public_addr":"172.21.15.170:6801/2448510618","cluster_addr":"172.21.15.170:6803/2448510618","heartbeat_back_addr":"172.21.15.170:6807/2448510618","heartbeat_front_addr":"172.21.15.170:6805/2448510618","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:20.565497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:44.727200+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:15:08.391885+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/2092713821":"2024-02-14T14:12:43.611879+0000","172.21.15.55:0/4039921225":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3229618965":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/2979874222":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/1289822098":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3950769590":"2024-02-14T14:12:15.949949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: Detected new or changed devices on smithi170 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: Adjusting osd_memory_target on smithi170 to 17088M 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:15:13.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:13 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: Detected new or changed devices on smithi170 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: Adjusting osd_memory_target on smithi170 to 17088M 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: osdmap e19: 3 total, 3 up, 3 in 2024-02-13T14:15:13.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:13 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-13T14:15:13.885 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49701]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-13T14:15:13.886 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49701]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:13.886 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49701]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:13.886 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49701]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:14.222 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49721]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-13T14:15:14.222 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49721]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:14.222 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49721]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:14.222 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:15:13 smithi055 sudo[49721]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:14.252 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-13T14:15:11.960022+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-13T14:15:14.252 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-13T14:15:14.408 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:14 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:14.408 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:14 smithi078 ceph-mon[30086]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:15:14.408 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:14 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/1622314990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:14.408 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:14 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:14.408 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:14 smithi078 ceph-mon[30086]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:15:14.408 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40304]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-13T14:15:14.408 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40304]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:14.408 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40304]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:14.408 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40304]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:14.497 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:14.564 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:14.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:14 smithi055 ceph-mon[25769]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:15:14.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:14 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/1622314990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:14.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:14 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:14.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:14 smithi055 ceph-mon[25769]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:15:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:14 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-13T14:15:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:14 smithi170 ceph-mon[30181]: osdmap e20: 3 total, 3 up, 3 in 2024-02-13T14:15:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:14 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/1622314990' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:14 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:14 smithi170 ceph-mon[30181]: osdmap e21: 3 total, 3 up, 3 in 2024-02-13T14:15:14.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40307]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-13T14:15:14.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40307]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:14.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40307]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:14.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:15:14 smithi078 sudo[40307]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:14.923 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40504]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-13T14:15:14.923 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40504]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:14.924 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40504]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:14.924 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40504]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:15.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40507]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-13T14:15:15.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40507]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:15.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40507]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:15.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:15:14 smithi170 sudo[40507]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:15.489 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49838]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:15:15.489 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49838]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:15.489 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49838]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:15.489 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49838]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:15.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 ceph-mon[30181]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:15.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.759 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49849]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49849]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 sudo[49849]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 ceph-mon[25769]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.811 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:15 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:15.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:15 smithi078 ceph-mon[30086]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:15.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:15 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:15 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:15.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:15 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:16.009 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 sudo[40510]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:15:16.009 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 sudo[40510]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:16.009 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 sudo[40510]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:16.009 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:15 smithi170 sudo[40510]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:16.375 INFO:teuthology.orchestra.run.smithi055.stdout:pg_num: 1 2024-02-13T14:15:16.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 sudo[40513]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:15:16.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 sudo[40513]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:16.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 sudo[40513]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:16.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 sudo[40513]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:16.523 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40310]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-13T14:15:16.523 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40310]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:16.523 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40310]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:16.523 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40310]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:16.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.686 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.687 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:16 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/484510216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:16.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:16 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/484510216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40313]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40313]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40313]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 sudo[40313]: pam_unix(sudo:session): session closed for user root 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:16.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:16.950 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:16 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/484510216' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-13T14:15:16.990 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi055 2024-02-13T14:15:16.990 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi055=foo.a' 2024-02-13T14:15:17.236 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:17.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:17 smithi170 ceph-mon[30181]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:17.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:17 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:17.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:17 smithi170 ceph-mon[30181]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:17.630 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:17 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:17 smithi055 ceph-mon[25769]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:17 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:17 smithi055 ceph-mon[25769]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:17.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:17 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:17 smithi078 ceph-mon[30086]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:17 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-13T14:15:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:17 smithi078 ceph-mon[30086]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-13T14:15:17.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:17 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:19.001 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled rgw.foo.a update... 2024-02-13T14:15:19.581 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:19 smithi170 ceph-mon[30181]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:19.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:19 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:19.582 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:19 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:19.607 DEBUG:teuthology.orchestra.run.smithi055:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service 2024-02-13T14:15:19.610 INFO:tasks.cephadm:Adding prometheus.a on smithi078 2024-02-13T14:15:19.610 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply prometheus '1;smithi078=a' 2024-02-13T14:15:19.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:19 smithi055 ceph-mon[25769]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:19.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:19 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:19.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:19 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:19.846 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:19.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:19 smithi078 ceph-mon[30086]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 85 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:19.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:19 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:19.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:19 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi055=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:20.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.816 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:20 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi055=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:20 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi055=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-13T14:15:20.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-13T14:15:20.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:20.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:20 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:21.605 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled prometheus update... 2024-02-13T14:15:21.743 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:21 smithi055 ceph-mon[25769]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:21.743 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:21 smithi055 ceph-mon[25769]: Deploying daemon rgw.foo.a on smithi055 2024-02-13T14:15:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:21 smithi170 ceph-mon[30181]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:21 smithi170 ceph-mon[30181]: Deploying daemon rgw.foo.a on smithi055 2024-02-13T14:15:21.947 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:21 smithi078 ceph-mon[30086]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:21.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:21 smithi078 ceph-mon[30086]: Deploying daemon rgw.foo.a on smithi055 2024-02-13T14:15:22.065 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:15:21 smithi055 podman[50250]: 2024-02-13 14:15:21.964109639 +0000 UTC m=+0.122219586 container create b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-13T14:15:22.211 DEBUG:teuthology.orchestra.run.smithi078:prometheus.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@prometheus.a.service 2024-02-13T14:15:22.214 INFO:tasks.cephadm:Adding node-exporter.a on smithi055 2024-02-13T14:15:22.214 INFO:tasks.cephadm:Adding node-exporter.b on smithi078 2024-02-13T14:15:22.215 INFO:tasks.cephadm:Adding node-exporter.c on smithi170 2024-02-13T14:15:22.215 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply node-exporter '3;smithi055=a;smithi078=b;smithi170=c' 2024-02-13T14:15:22.451 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:22.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:15:22 smithi055 podman[50250]: 2024-02-13 14:15:22.199837703 +0000 UTC m=+0.357947609 container init b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:15:22.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:15:22 smithi055 podman[50250]: 2024-02-13 14:15:22.205454225 +0000 UTC m=+0.363564130 container start b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:15:22.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:15:22 smithi055 bash[50250]: b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 2024-02-13T14:15:22.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:15:22 smithi055 systemd[1]: Started Ceph rgw.foo.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: Saving service prometheus spec with placement smithi078=a;count:1 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.862 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:22 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: Saving service prometheus spec with placement smithi078=a;count:1 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:22.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:22 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: Saving service prometheus spec with placement smithi078=a;count:1 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:22 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:23.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:23 smithi170 ceph-mon[30181]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:23.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:23 smithi170 ceph-mon[30181]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:23.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:23 smithi170 ceph-mon[30181]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:15:23.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:23 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:23.765 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:23 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:23.778 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:23 smithi055 ceph-mon[25769]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:23.779 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:23 smithi055 ceph-mon[25769]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:23.779 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:23 smithi055 ceph-mon[25769]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:15:23.779 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:23 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:23.779 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:23 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:23.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:23 smithi078 ceph-mon[30086]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:23.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:23 smithi078 ceph-mon[30086]: Saving service rgw.foo.a spec with placement smithi055=foo.a;count:1 2024-02-13T14:15:23.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:23 smithi078 ceph-mon[30086]: osdmap e22: 3 total, 3 up, 3 in 2024-02-13T14:15:23.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:23 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-13T14:15:23.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:23 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:24.209 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled node-exporter update... 2024-02-13T14:15:24.813 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: osdmap e23: 3 total, 3 up, 3 in 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: pgmap v93: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi055=a;smithi078=b;smithi170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: Saving service node-exporter spec with placement smithi055=a;smithi078=b;smithi170=c;count:3 2024-02-13T14:15:24.814 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:24 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.841 DEBUG:teuthology.orchestra.run.smithi055:node-exporter.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.a.service 2024-02-13T14:15:24.844 DEBUG:teuthology.orchestra.run.smithi078:node-exporter.b> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.b.service 2024-02-13T14:15:24.847 DEBUG:teuthology.orchestra.run.smithi170:node-exporter.c> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.c.service 2024-02-13T14:15:24.849 INFO:tasks.cephadm:Adding alertmanager.a on smithi055 2024-02-13T14:15:24.850 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply alertmanager '1;smithi055=a' 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: osdmap e23: 3 total, 3 up, 3 in 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: pgmap v93: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:24.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:24.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:24.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi055=a;smithi078=b;smithi170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:24.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: Saving service node-exporter spec with placement smithi055=a;smithi078=b;smithi170=c;count:3 2024-02-13T14:15:24.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:24 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/85045037' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: osdmap e23: 3 total, 3 up, 3 in 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: pgmap v93: 33 pgs: 18 creating+peering, 14 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:15:25.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: Checking dashboard <-> RGW credentials 2024-02-13T14:15:25.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi055=a;smithi078=b;smithi170=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:25.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: Saving service node-exporter spec with placement smithi055=a;smithi078=b;smithi170=c;count:3 2024-02-13T14:15:25.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:24 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:25.121 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:25 smithi170 ceph-mon[30181]: osdmap e24: 3 total, 3 up, 3 in 2024-02-13T14:15:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:25 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:25.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:25 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:25.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:25 smithi078 ceph-mon[30086]: osdmap e24: 3 total, 3 up, 3 in 2024-02-13T14:15:25.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:25 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:25.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:25 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:26.064 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:25 smithi055 ceph-mon[25769]: osdmap e24: 3 total, 3 up, 3 in 2024-02-13T14:15:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:25 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:26.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:25 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-13T14:15:26.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:26 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:26.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:26 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:26.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:26 smithi170 ceph-mon[30181]: osdmap e25: 3 total, 3 up, 3 in 2024-02-13T14:15:26.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:26 smithi170 ceph-mon[30181]: pgmap v96: 65 pgs: 18 creating+peering, 46 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:26.779 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:26 smithi170 ceph-mon[30181]: osdmap e26: 3 total, 3 up, 3 in 2024-02-13T14:15:26.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:26 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:26.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:26 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:26.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:26 smithi078 ceph-mon[30086]: osdmap e25: 3 total, 3 up, 3 in 2024-02-13T14:15:26.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:26 smithi078 ceph-mon[30086]: pgmap v96: 65 pgs: 18 creating+peering, 46 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:26.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:26 smithi078 ceph-mon[30086]: osdmap e26: 3 total, 3 up, 3 in 2024-02-13T14:15:26.958 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled alertmanager update... 2024-02-13T14:15:27.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:26 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:27.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:26 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-13T14:15:27.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:26 smithi055 ceph-mon[25769]: osdmap e25: 3 total, 3 up, 3 in 2024-02-13T14:15:27.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:26 smithi055 ceph-mon[25769]: pgmap v96: 65 pgs: 18 creating+peering, 46 unknown, 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:27.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:26 smithi055 ceph-mon[25769]: osdmap e26: 3 total, 3 up, 3 in 2024-02-13T14:15:27.603 DEBUG:teuthology.orchestra.run.smithi055:alertmanager.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@alertmanager.a.service 2024-02-13T14:15:27.606 INFO:tasks.cephadm:Adding grafana.a on smithi078 2024-02-13T14:15:27.606 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph orch apply grafana '1;smithi078=a' 2024-02-13T14:15:27.868 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:27.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:27.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:27.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi055=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:27.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: Saving service alertmanager spec with placement smithi055=a;count:1 2024-02-13T14:15:27.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:27.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:27.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:27.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:27 smithi170 ceph-mon[30181]: osdmap e27: 3 total, 3 up, 3 in 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi055=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: Saving service alertmanager spec with placement smithi055=a;count:1 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:27.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:27.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:27.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:27 smithi078 ceph-mon[30086]: osdmap e27: 3 total, 3 up, 3 in 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi055=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: Saving service alertmanager spec with placement smithi055=a;count:1 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-13T14:15:28.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:27 smithi055 ceph-mon[25769]: osdmap e27: 3 total, 3 up, 3 in 2024-02-13T14:15:28.898 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:28 smithi170 ceph-mon[30181]: pgmap v99: 97 pgs: 17 creating+peering, 35 unknown, 45 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:28.899 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:28 smithi170 ceph-mon[30181]: osdmap e28: 3 total, 3 up, 3 in 2024-02-13T14:15:28.899 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:28 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:28.899 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:28 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:28.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:28 smithi078 ceph-mon[30086]: pgmap v99: 97 pgs: 17 creating+peering, 35 unknown, 45 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:28.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:28 smithi078 ceph-mon[30086]: osdmap e28: 3 total, 3 up, 3 in 2024-02-13T14:15:28.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:28 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:28.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:28 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:29.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:28 smithi055 ceph-mon[25769]: pgmap v99: 97 pgs: 17 creating+peering, 35 unknown, 45 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-13T14:15:29.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:28 smithi055 ceph-mon[25769]: osdmap e28: 3 total, 3 up, 3 in 2024-02-13T14:15:29.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:28 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:29.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:28 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-13T14:15:29.619 INFO:teuthology.orchestra.run.smithi170.stdout:Scheduled grafana update... 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: osdmap e29: 3 total, 3 up, 3 in 2024-02-13T14:15:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:29.907 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:29 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: osdmap e29: 3 total, 3 up, 3 in 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:29.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:29 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:30.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi078=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:15:30.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:30.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:30.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-13T14:15:30.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: osdmap e29: 3 total, 3 up, 3 in 2024-02-13T14:15:30.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:30.066 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:29 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-13T14:15:30.132 DEBUG:teuthology.orchestra.run.smithi078:grafana.a> sudo journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@grafana.a.service 2024-02-13T14:15:30.136 INFO:tasks.cephadm:Setting up client nodes... 2024-02-13T14:15:30.136 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T14:15:30.388 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:30.800 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:30 smithi055 ceph-mon[25769]: Saving service grafana spec with placement smithi078=a;count:1 2024-02-13T14:15:30.800 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:30 smithi055 ceph-mon[25769]: pgmap v102: 129 pgs: 9 creating+peering, 44 unknown, 76 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-13T14:15:30.800 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:30 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.800 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:30 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.800 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:30 smithi055 ceph-mon[25769]: osdmap e30: 3 total, 3 up, 3 in 2024-02-13T14:15:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:30 smithi170 ceph-mon[30181]: Saving service grafana spec with placement smithi078=a;count:1 2024-02-13T14:15:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:30 smithi170 ceph-mon[30181]: pgmap v102: 129 pgs: 9 creating+peering, 44 unknown, 76 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-13T14:15:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:30 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:30 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:30 smithi170 ceph-mon[30181]: osdmap e30: 3 total, 3 up, 3 in 2024-02-13T14:15:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:30 smithi078 ceph-mon[30086]: Saving service grafana spec with placement smithi078=a;count:1 2024-02-13T14:15:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:30 smithi078 ceph-mon[30086]: pgmap v102: 129 pgs: 9 creating+peering, 44 unknown, 76 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-13T14:15:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:30 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/481591266' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:30 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3456909416' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-13T14:15:30.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:30 smithi078 ceph-mon[30086]: osdmap e30: 3 total, 3 up, 3 in 2024-02-13T14:15:32.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:31 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:31 smithi170 ceph-mon[30181]: Deploying daemon prometheus.a on smithi078 2024-02-13T14:15:32.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:31 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:31 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:31 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:31 smithi078 ceph-mon[30086]: Deploying daemon prometheus.a on smithi078 2024-02-13T14:15:32.314 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:31 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:31 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:31 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:32.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:31 smithi055 ceph-mon[25769]: Deploying daemon prometheus.a on smithi078 2024-02-13T14:15:32.405 INFO:teuthology.orchestra.run.smithi055.stdout:[client.0] 2024-02-13T14:15:32.405 INFO:teuthology.orchestra.run.smithi055.stdout: key = AQCEectll+fmFxAAjSpSovNcDWD9BIavgtIqsw== 2024-02-13T14:15:32.968 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:32 smithi055 ceph-mon[25769]: pgmap v104: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 4.2 KiB/s wr, 82 op/s 2024-02-13T14:15:32.968 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:32 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:32.968 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:32 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:32.995 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:15:32.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-13T14:15:32.995 DEBUG:teuthology.orchestra.run.smithi055:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-13T14:15:33.038 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T14:15:33.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:32 smithi170 ceph-mon[30181]: pgmap v104: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 4.2 KiB/s wr, 82 op/s 2024-02-13T14:15:33.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:32 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:33.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:32 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:33.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:32 smithi078 ceph-mon[30086]: pgmap v104: 129 pgs: 9 creating+peering, 23 unknown, 97 active+clean; 579 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 35 KiB/s rd, 4.2 KiB/s wr, 82 op/s 2024-02-13T14:15:33.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:32 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:33.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:32 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2320472515' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:33.489 INFO:teuthology.orchestra.run.smithi078.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.b/config 2024-02-13T14:15:35.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:34 smithi170 ceph-mon[30181]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 79 KiB/s rd, 6.7 KiB/s wr, 190 op/s 2024-02-13T14:15:35.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:34 smithi078 ceph-mon[30086]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 79 KiB/s rd, 6.7 KiB/s wr, 190 op/s 2024-02-13T14:15:35.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:34 smithi055 ceph-mon[25769]: pgmap v105: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 79 KiB/s rd, 6.7 KiB/s wr, 190 op/s 2024-02-13T14:15:36.310 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2024-02-13T14:15:36.310 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQCIectl/k8JEhAA4Pr64mtUN1r1Sesw19VTSw== 2024-02-13T14:15:37.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:36 smithi170 ceph-mon[30181]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 160 op/s 2024-02-13T14:15:37.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:36 smithi170 ceph-mon[30181]: from='client.? 172.21.15.78:0/3937592146' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:36 smithi170 ceph-mon[30181]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:36 smithi170 ceph-mon[30181]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:37.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:36 smithi078 ceph-mon[30086]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 160 op/s 2024-02-13T14:15:37.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:36 smithi078 ceph-mon[30086]: from='client.? 172.21.15.78:0/3937592146' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:36 smithi078 ceph-mon[30086]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:36 smithi078 ceph-mon[30086]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:37.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:36 smithi055 ceph-mon[25769]: pgmap v106: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.0 KiB/s wr, 160 op/s 2024-02-13T14:15:37.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:36 smithi055 ceph-mon[25769]: from='client.? 172.21.15.78:0/3937592146' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:36 smithi055 ceph-mon[25769]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:37.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:36 smithi055 ceph-mon[25769]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:38.071 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:15:38.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-13T14:15:38.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-13T14:15:38.116 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-13T14:15:38.355 INFO:teuthology.orchestra.run.smithi170.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.c/config 2024-02-13T14:15:39.117 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:38 smithi055 ceph-mon[25769]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-13T14:15:39.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:38 smithi170 ceph-mon[30181]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-13T14:15:39.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:38 smithi078 ceph-mon[30086]: pgmap v107: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.4 KiB/s wr, 140 op/s 2024-02-13T14:15:40.266 INFO:teuthology.orchestra.run.smithi170.stdout:[client.2] 2024-02-13T14:15:40.266 INFO:teuthology.orchestra.run.smithi170.stdout: key = AQCMectln3GIDxAAsqF74FhGYu2cUkkIWuei+Q== 2024-02-13T14:15:40.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:40 smithi170 ceph-mon[30181]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-13T14:15:40.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:40 smithi170 ceph-mon[30181]: from='client.? 172.21.15.170:0/2855258960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:40.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:40 smithi170 ceph-mon[30181]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:40.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:40 smithi170 ceph-mon[30181]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:40.933 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:40 smithi078 ceph-mon[30086]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-13T14:15:40.934 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:40 smithi078 ceph-mon[30086]: from='client.? 172.21.15.170:0/2855258960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:40.934 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:40 smithi078 ceph-mon[30086]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:40.934 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:40 smithi078 ceph-mon[30086]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:40.943 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:15:40.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-13T14:15:40.943 DEBUG:teuthology.orchestra.run.smithi170:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-13T14:15:40.984 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-13T14:15:40.984 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-13T14:15:40.985 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph mgr dump --format=json 2024-02-13T14:15:41.239 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:41.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:40 smithi055 ceph-mon[25769]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-02-13T14:15:41.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:40 smithi055 ceph-mon[25769]: from='client.? 172.21.15.170:0/2855258960' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:41.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:40 smithi055 ceph-mon[25769]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-13T14:15:41.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:40 smithi055 ceph-mon[25769]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-13T14:15:42.448 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 systemd[1]: Starting Ceph prometheus.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:15:42.746 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 podman[41219]: 2024-02-13 14:15:42.382201832 +0000 UTC m=+0.018359496 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-13T14:15:42.746 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 podman[41219]: 2024-02-13 14:15:42.481221465 +0000 UTC m=+0.117379159 container create 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:42.746 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 podman[41219]: 2024-02-13 14:15:42.716660053 +0000 UTC m=+0.352817707 container init 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:42.747 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 podman[41219]: 2024-02-13 14:15:42.721731472 +0000 UTC m=+0.357889124 container start 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:15:43.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:42 smithi055 ceph-mon[25769]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-02-13T14:15:43.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:42 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:43.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:42 smithi170 ceph-mon[30181]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-02-13T14:15:43.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:42 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:43.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-mon[30086]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.2 KiB/s wr, 102 op/s 2024-02-13T14:15:43.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:43.198 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.745Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.745Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.745Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi078 (none))" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.745Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.745Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.747Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.747Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.748Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.748Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.750Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.750Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.089µs 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.750Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.751Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.751Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=33.856µs wal_replay_duration=521.993µs wbl_replay_duration=130ns total_replay_duration=616.892µs 2024-02-13T14:15:43.199 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.752Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.752Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.752Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.769Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=17.32947ms db_storage=697ns remote_storage=973ns web_handler=372ns query_engine=508ns scrape=440.858µs scrape_sd=67.602µs notify=1.09µs notify_sd=1.35µs rules=16.63876ms tracing=5.741µs 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.769Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:15:42.769Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 bash[41219]: 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 2024-02-13T14:15:43.200 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:15:42 smithi078 systemd[1]: Started Ceph prometheus.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:15:43.205 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:15:43.994 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6800","nonce":1133021869},{"type":"v1","addr":"172.21.15.55:6801","nonce":1133021869}]},"active_addr":"172.21.15.55:6801/1133021869","active_change":"2024-02-13T14:12:43.611995+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.55:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":4027152363}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":993734249}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":854277738}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.55:0","nonce":474786498}]}]} 2024-02-13T14:15:44.000 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-13T14:15:44.000 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-13T14:15:44.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:15:44.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:44.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3935489704' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:44.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: ignoring --setuser ceph since I am not root 2024-02-13T14:15:44.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: ignoring --setgroup ceph since I am not root 2024-02-13T14:15:44.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:43 smithi170 ceph-mgr[32292]: -- 172.21.15.170:0/3546960239 <== mon.1 v2:172.21.15.170:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55fd2285c1a0 con 0x55fd22833000 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: ignoring --setuser ceph since I am not root 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: ignoring --setgroup ceph since I am not root 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-mgr[31564]: -- 172.21.15.78:0/233042991 <== mon.2 v2:172.21.15.78:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d4a39681a0 con 0x55d4a393e800 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:44.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:43 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3935489704' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:44.255 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3935489704' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setuser ceph since I am not root 2024-02-13T14:15:44.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ignoring --setgroup ceph since I am not root 2024-02-13T14:15:44.667 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:44.516+0000 7fed76d5c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:15:44.667 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:44.615+0000 7fed76d5c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:15:44.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:44 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:44.468+0000 7f8f717bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:15:44.906 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:44 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:44.562+0000 7f8f717bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:15:44.907 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:44 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:44.471+0000 7f61154bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-13T14:15:44.908 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:44 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:44.565+0000 7f61154bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-13T14:15:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:44 smithi055 ceph-mon[25769]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:45.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:44 smithi055 ceph-mon[25769]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-13T14:15:45.194 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:44 smithi170 ceph-mon[30181]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:45.195 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:44 smithi170 ceph-mon[30181]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-13T14:15:45.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:44 smithi078 ceph-mon[30086]: from='mgr.14150 172.21.15.55:0/2702360642' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-13T14:15:45.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:44 smithi078 ceph-mon[30086]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-13T14:15:45.374 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.277+0000 7fed76d5c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:15:45.524 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.192+0000 7f8f717bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:15:45.524 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.286+0000 7f8f717bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:15:45.524 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.427+0000 7f8f717bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:15:45.524 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.522+0000 7f8f717bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:15:45.535 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.203+0000 7f61154bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-13T14:15:45.535 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.296+0000 7f61154bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:15:45.535 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.437+0000 7f61154bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:15:45.536 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.533+0000 7f61154bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:15:45.689 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.373+0000 7fed76d5c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-13T14:15:45.690 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.519+0000 7fed76d5c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-13T14:15:45.690 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.619+0000 7fed76d5c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-13T14:15:45.871 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.605+0000 7f8f717bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:15:45.871 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.869+0000 7f8f717bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:15:45.882 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.618+0000 7f61154bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:15:45.987 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.708+0000 7fed76d5c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-13T14:15:45.987 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:45 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:45.985+0000 7fed76d5c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:15:46.146 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:15:46.146 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":30,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:11:43.841892+0000","modified":"2024-02-13T14:15:30.633316+0000","last_up_change":"2024-02-13T14:15:10.530332+0000","last_in_change":"2024-02-13T14:14:51.816199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:15:11.960022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-13T14:15:22.484042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-13T14:15:23.935024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-13T14:15:25.891245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-13T14:15:27.943944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3667302d-4268-42ea-9250-c1a648747a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6803","nonce":2907918115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6805","nonce":2907918115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6809","nonce":2907918115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6807","nonce":2907918115}]},"public_addr":"172.21.15.55:6803/2907918115","cluster_addr":"172.21.15.55:6805/2907918115","heartbeat_back_addr":"172.21.15.55:6809/2907918115","heartbeat_front_addr":"172.21.15.55:6807/2907918115","state":["exists","up"]},{"osd":1,"uuid":"729b1700-978e-4a40-a405-13b420956686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6801","nonce":3405515124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6803","nonce":3405515124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6807","nonce":3405515124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6805","nonce":3405515124}]},"public_addr":"172.21.15.78:6801/3405515124","cluster_addr":"172.21.15.78:6803/3405515124","heartbeat_back_addr":"172.21.15.78:6807/3405515124","heartbeat_front_addr":"172.21.15.78:6805/3405515124","state":["exists","up"]},{"osd":2,"uuid":"1cd8c691-2f2e-4edf-a116-b96a87f2c815","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6801","nonce":2448510618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6803","nonce":2448510618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6807","nonce":2448510618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6805","nonce":2448510618}]},"public_addr":"172.21.15.170:6801/2448510618","cluster_addr":"172.21.15.170:6803/2448510618","heartbeat_back_addr":"172.21.15.170:6807/2448510618","heartbeat_front_addr":"172.21.15.170:6805/2448510618","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:20.565497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:44.727200+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:15:08.391885+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/2092713821":"2024-02-14T14:12:43.611879+0000","172.21.15.55:0/4039921225":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3229618965":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/2979874222":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/1289822098":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3950769590":"2024-02-14T14:12:15.949949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:46.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:45 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:45.977+0000 7f8f717bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:46.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:46 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:46.063+0000 7f8f717bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:15:46.171 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.880+0000 7f61154bd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-13T14:15:46.171 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:45.990+0000 7f61154bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:46.171 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:46 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:46.075+0000 7f61154bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:15:46.171 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:46 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:46.169+0000 7f61154bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:15:46.292 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:46 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/4161578062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:46.293 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:46 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:46.098+0000 7fed76d5c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:46.293 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:46 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:46.192+0000 7fed76d5c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-13T14:15:46.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:46 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/4161578062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:46.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:46 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:46.345+0000 7f61154bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:15:46.564 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:46 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:46.291+0000 7fed76d5c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:15:46.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:46 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:46.470+0000 7fed76d5c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:15:46.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:46 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/4161578062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:46.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:46 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:46.157+0000 7f8f717bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-13T14:15:46.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:46 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:46.334+0000 7f8f717bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-13T14:15:46.855 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-13T14:15:46.855 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd dump --format=json 2024-02-13T14:15:47.116 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:47.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:47 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:47.287+0000 7f8f717bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:15:47.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:47 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:47.416+0000 7f8f717bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:15:47.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:47 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:47.305+0000 7f61154bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:15:47.698 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:47 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:47.436+0000 7f61154bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:15:47.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:47.490+0000 7fed76d5c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-13T14:15:47.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:47.630+0000 7fed76d5c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-13T14:15:48.035 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:47 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:47.748+0000 7f8f717bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:15:48.035 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:47 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:47.838+0000 7f8f717bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:15:48.035 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:48 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:48.032+0000 7f8f717bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:48.057 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:47 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:47.769+0000 7f61154bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:15:48.057 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:47 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:47.860+0000 7f61154bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:15:48.057 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:48 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:48.055+0000 7f61154bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:48.258 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:47.973+0000 7fed76d5c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-13T14:15:48.259 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:48 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:48.067+0000 7fed76d5c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-13T14:15:48.406 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:48 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:48.214+0000 7f8f717bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:15:48.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:48 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:48.240+0000 7f61154bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:15:48.564 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:48 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:48.271+0000 7fed76d5c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:48.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:48 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:48.467+0000 7fed76d5c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-13T14:15:48.760 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:48 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:48.496+0000 7f8f717bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:48.761 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:48 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:48.758+0000 7f8f717bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:48.783 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:48 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:48.525+0000 7f61154bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:49.041 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:48 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:48.772+0000 7fed76d5c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-13T14:15:49.041 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:49 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:49.039+0000 7fed76d5c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:49.042 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:15:49.042 INFO:teuthology.orchestra.run.smithi055.stdout:{"epoch":30,"fsid":"9e367004-ca79-11ee-95b9-87774f69a715","created":"2024-02-13T14:11:43.841892+0000","modified":"2024-02-13T14:15:30.633316+0000","last_up_change":"2024-02-13T14:15:10.530332+0000","last_in_change":"2024-02-13T14:14:51.816199+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-13T14:15:11.960022+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-13T14:15:22.484042+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-13T14:15:23.935024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-13T14:15:25.891245+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-13T14:15:27.943944+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3667302d-4268-42ea-9250-c1a648747a0b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6802","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6803","nonce":2907918115}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6804","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6805","nonce":2907918115}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6808","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6809","nonce":2907918115}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.55:6806","nonce":2907918115},{"type":"v1","addr":"172.21.15.55:6807","nonce":2907918115}]},"public_addr":"172.21.15.55:6803/2907918115","cluster_addr":"172.21.15.55:6805/2907918115","heartbeat_back_addr":"172.21.15.55:6809/2907918115","heartbeat_front_addr":"172.21.15.55:6807/2907918115","state":["exists","up"]},{"osd":1,"uuid":"729b1700-978e-4a40-a405-13b420956686","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6801","nonce":3405515124}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6803","nonce":3405515124}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6807","nonce":3405515124}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":3405515124},{"type":"v1","addr":"172.21.15.78:6805","nonce":3405515124}]},"public_addr":"172.21.15.78:6801/3405515124","cluster_addr":"172.21.15.78:6803/3405515124","heartbeat_back_addr":"172.21.15.78:6807/3405515124","heartbeat_front_addr":"172.21.15.78:6805/3405515124","state":["exists","up"]},{"osd":2,"uuid":"1cd8c691-2f2e-4edf-a116-b96a87f2c815","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6800","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6801","nonce":2448510618}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6802","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6803","nonce":2448510618}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6806","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6807","nonce":2448510618}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.170:6804","nonce":2448510618},{"type":"v1","addr":"172.21.15.170:6805","nonce":2448510618}]},"public_addr":"172.21.15.170:6801/2448510618","cluster_addr":"172.21.15.170:6803/2448510618","heartbeat_back_addr":"172.21.15.170:6807/2448510618","heartbeat_front_addr":"172.21.15.170:6805/2448510618","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:20.565497+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:14:44.727200+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-13T14:15:08.391885+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.55:0/2092713821":"2024-02-14T14:12:43.611879+0000","172.21.15.55:0/4039921225":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/2172131115":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6800/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3229618965":"2024-02-14T14:12:43.611879+0000","172.21.15.55:6801/421369985":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/2979874222":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/1289822098":"2024-02-14T14:12:15.949949+0000","172.21.15.55:0/3950769590":"2024-02-14T14:12:15.949949+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-13T14:15:49.093 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:48 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:48.992+0000 7f8f717bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:15:49.094 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:48 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:48.778+0000 7f61154bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-13T14:15:49.094 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:49 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:49.011+0000 7f61154bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:15:49.315 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:49 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2913216312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:49.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:49 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:49.288+0000 7fed76d5c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-13T14:15:49.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:49 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2913216312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:49.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:49 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2913216312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-13T14:15:49.902 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-13T14:15:49.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-13T14:15:49.903 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-13T14:15:50.156 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:49 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:49.782+0000 7f8f717bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:15:50.173 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:50.197 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:49 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:49.802+0000 7f61154bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:15:50.202 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:50.203 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:50.438 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:50.169+0000 7fed76d5c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-13T14:15:50.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:50.352+0000 7f8f717bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:15:50.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:50.449+0000 7f8f717bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:15:50.673 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:50.363+0000 7f61154bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:15:50.673 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:50.459+0000 7f61154bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:50.671+0000 7f61154bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: [13/Feb/2024:14:15:50] ENGINE Bus STARTING 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: CherryPy Checker: 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: The Application mounted at '' has an empty config. 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: [13/Feb/2024:14:15:50] ENGINE Serving on http://:::9283 2024-02-13T14:15:50.948 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x[31540]: [13/Feb/2024:14:15:50] ENGINE Bus STARTED 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: Standby manager daemon z restarted 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: Standby manager daemon z started 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: Standby manager daemon x restarted 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: Standby manager daemon x started 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:50.949 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:50.950 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:50.950 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:50.950 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:50 smithi078 ceph-mon[30086]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:51.007 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:50.785+0000 7fed76d5c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-13T14:15:51.007 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:50.887+0000 7fed76d5c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-13T14:15:51.007 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: Standby manager daemon z restarted 2024-02-13T14:15:51.007 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: Standby manager daemon z started 2024-02-13T14:15:51.007 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: Standby manager daemon x restarted 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: Standby manager daemon x started 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:51.008 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:50 smithi055 ceph-mon[25769]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:51.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: Standby manager daemon z restarted 2024-02-13T14:15:51.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: Standby manager daemon z started 2024-02-13T14:15:51.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-13T14:15:51.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: Standby manager daemon x restarted 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: Standby manager daemon x started 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.170:0/338710752' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-mon[30181]: from='mgr.? 172.21.15.78:0/3340582978' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:50.666+0000 7f8f717bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:15:51.157 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: [13/Feb/2024:14:15:50] ENGINE Bus STARTING 2024-02-13T14:15:51.158 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: CherryPy Checker: 2024-02-13T14:15:51.158 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: The Application mounted at '' has an empty config. 2024-02-13T14:15:51.158 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: 2024-02-13T14:15:51.158 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: [13/Feb/2024:14:15:50] ENGINE Serving on http://:::9283 2024-02-13T14:15:51.158 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:15:50 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z[32269]: [13/Feb/2024:14:15:50] ENGINE Bus STARTED 2024-02-13T14:15:51.294 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:51.124+0000 7fed76d5c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-13T14:15:51.294 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:15:51] ENGINE Bus STARTING 2024-02-13T14:15:51.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: CherryPy Checker: 2024-02-13T14:15:51.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: The Application mounted at '' has an empty config. 2024-02-13T14:15:51.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:15:51.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:15:51] ENGINE Serving on http://:::9283 2024-02-13T14:15:51.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:15:51] ENGINE Bus STARTED 2024-02-13T14:15:51.870 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-13T14:15:51.870 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: Active manager daemon y restarted 2024-02-13T14:15:51.870 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: Activating manager daemon y 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: osdmap e31: 3 total, 3 up, 3 in 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: mgrmap e19: y(active, starting, since 0.0137045s), standbys: z, x 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:51.871 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: Manager daemon y is now available 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:51.872 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:51 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:52.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-13T14:15:52.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: Active manager daemon y restarted 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: Activating manager daemon y 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: osdmap e31: 3 total, 3 up, 3 in 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: mgrmap e19: y(active, starting, since 0.0137045s), standbys: z, x 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:52.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: Manager daemon y is now available 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:52.158 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:51 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:52.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: mgrmap e18: y(active, since 3m), standbys: z, x 2024-02-13T14:15:52.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: Active manager daemon y restarted 2024-02-13T14:15:52.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: Activating manager daemon y 2024-02-13T14:15:52.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: osdmap e31: 3 total, 3 up, 3 in 2024-02-13T14:15:52.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: mgrmap e19: y(active, starting, since 0.0137045s), standbys: z, x 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: Manager daemon y is now available 2024-02-13T14:15:52.199 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:52.200 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-13T14:15:52.200 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:52.200 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:15:52.200 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-13T14:15:52.200 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:51 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-13T14:15:53.059 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:52 smithi055 ceph-mon[25769]: [13/Feb/2024:14:15:51] ENGINE Bus STARTING 2024-02-13T14:15:53.059 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:52 smithi055 ceph-mon[25769]: mgrmap e20: y(active, since 1.01787s), standbys: z, x 2024-02-13T14:15:53.156 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:52 smithi078 ceph-mon[30086]: [13/Feb/2024:14:15:51] ENGINE Bus STARTING 2024-02-13T14:15:53.156 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:52 smithi078 ceph-mon[30086]: mgrmap e20: y(active, since 1.01787s), standbys: z, x 2024-02-13T14:15:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:52 smithi170 ceph-mon[30181]: [13/Feb/2024:14:15:51] ENGINE Bus STARTING 2024-02-13T14:15:53.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:52 smithi170 ceph-mon[30181]: mgrmap e20: y(active, since 1.01787s), standbys: z, x 2024-02-13T14:15:53.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:15:53] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:15:53.985 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: [13/Feb/2024:14:15:51] ENGINE Serving on https://172.21.15.55:7150 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: [13/Feb/2024:14:15:52] ENGINE Serving on http://172.21.15.55:8765 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: [13/Feb/2024:14:15:52] ENGINE Bus STARTED 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:53.986 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:53 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: [13/Feb/2024:14:15:51] ENGINE Serving on https://172.21.15.55:7150 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: [13/Feb/2024:14:15:52] ENGINE Serving on http://172.21.15.55:8765 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: [13/Feb/2024:14:15:52] ENGINE Bus STARTED 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.157 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:53 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: [13/Feb/2024:14:15:51] ENGINE Serving on https://172.21.15.55:7150 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: [13/Feb/2024:14:15:52] ENGINE Serving on http://172.21.15.55:8765 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: [13/Feb/2024:14:15:52] ENGINE Bus STARTED 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:53 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:54.998 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:54 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:55.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:54 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:15:55.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:54 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:15:55.689 INFO:teuthology.orchestra.run.smithi055.stdout:34359738389 2024-02-13T14:15:55.690 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-13T14:15:55.963 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:55 smithi055 ceph-mon[25769]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-13T14:15:56.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:55 smithi170 ceph-mon[30181]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-13T14:15:56.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:55 smithi078 ceph-mon[30086]: Adjusting osd_memory_target on smithi078 to 16064M 2024-02-13T14:15:56.273 INFO:teuthology.orchestra.run.smithi055.stdout:55834574864 2024-02-13T14:15:56.274 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-13T14:15:56.911 INFO:teuthology.orchestra.run.smithi055.stdout:77309411339 2024-02-13T14:15:56.911 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-13T14:15:57.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:56 smithi170 ceph-mon[30181]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-13T14:15:57.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:56 smithi170 ceph-mon[30181]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-13T14:15:57.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:56 smithi055 ceph-mon[25769]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-13T14:15:57.178 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:56 smithi055 ceph-mon[25769]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-13T14:15:57.197 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:56 smithi078 ceph-mon[30086]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-13T14:15:57.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:56 smithi078 ceph-mon[30086]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-13T14:15:57.393 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:58.060 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:58.097 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:15:59.065 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:15:58 smithi055 ceph-mon[25769]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-13T14:15:59.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:15:58 smithi170 ceph-mon[30181]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-13T14:15:59.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:15:58 smithi078 ceph-mon[30086]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-13T14:16:01.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:00 smithi078 ceph-mon[30086]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-13T14:16:01.202 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:00 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:01.202 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:00 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:01.275 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:00 smithi055 ceph-mon[25769]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-13T14:16:01.276 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:00 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:01.276 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:00 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:01.364 INFO:teuthology.orchestra.run.smithi055.stdout:55834574865 2024-02-13T14:16:01.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:00 smithi170 ceph-mon[30181]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-13T14:16:01.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:00 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:01.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:00 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.124 INFO:teuthology.orchestra.run.smithi055.stdout:77309411340 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Adjusting osd_memory_target on smithi055 to 16069M 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:16:02.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/2986090625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:02 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/3003803284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:02.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-13T14:16:02.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Adjusting osd_memory_target on smithi055 to 16069M 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/2986090625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:02 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/3003803284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Adjusting osd_memory_target on smithi055 to 16069M 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi055:/etc/ceph/ceph.conf 2024-02-13T14:16:02.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi078:/etc/ceph/ceph.conf 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi170:/etc/ceph/ceph.conf 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/2986090625' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.conf 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi055:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi170:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: Updating smithi078:/etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:02.699 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:02 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/3003803284' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-13T14:16:02.923 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411339 got 77309411340 for osd.2 2024-02-13T14:16:02.923 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:03.147 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:16:03] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:03.387 INFO:teuthology.orchestra.run.smithi055.stdout:34359738391 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.400 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.401 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:03 smithi055 ceph-mon[25769]: Deploying daemon alertmanager.a on smithi055 2024-02-13T14:16:03.649 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574864 got 55834574865 for osd.1 2024-02-13T14:16:03.650 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:03.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:03 smithi170 ceph-mon[30181]: Deploying daemon alertmanager.a on smithi055 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: Updating smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: Updating smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: Updating smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/config/ceph.client.admin.keyring 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:03.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:03 smithi078 ceph-mon[30086]: Deploying daemon alertmanager.a on smithi055 2024-02-13T14:16:04.347 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738389 got 34359738391 for osd.0 2024-02-13T14:16:04.348 DEBUG:teuthology.parallel:result is None 2024-02-13T14:16:04.348 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-13T14:16:04.348 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T14:16:04.588 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:04.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:04 smithi170 ceph-mon[30181]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:04.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:04 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/1360175529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:04 smithi078 ceph-mon[30086]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:04.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:04 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/1360175529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:04.711 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:04 smithi055 ceph-mon[25769]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-13T14:16:04.711 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:04 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/1360175529' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-13T14:16:06.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:06 smithi055 ceph-mon[25769]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-13T14:16:06.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:06 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:06.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:06 smithi170 ceph-mon[30181]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-13T14:16:06.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:06 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:06.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:06 smithi078 ceph-mon[30086]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-13T14:16:06.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:06 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:07.444 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:16:07.446 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-02-13T14:16:08.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:08 smithi170 ceph-mon[30181]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:08.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:08 smithi170 ceph-mon[30181]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:08.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:08 smithi055 ceph-mon[25769]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:08.696 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:08 smithi055 ceph-mon[25769]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:08.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:08 smithi078 ceph-mon[30086]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:08.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:08 smithi078 ceph-mon[30086]: from='client.14514 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:08.725 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-02-13T14:16:07.160001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96616,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126552,"statfs":{"total":287972524032,"available":287873589248,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003201"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150719+0000","last_change":"2024-02-13T14:15:29.645879+0000","last_active":"2024-02-13T14:15:51.150719+0000","last_peered":"2024-02-13T14:15:51.150719+0000","last_clean":"2024-02-13T14:15:51.150719+0000","last_became_active":"2024-02-13T14:15:29.645337+0000","last_became_peered":"2024-02-13T14:15:29.645337+0000","last_unstale":"2024-02-13T14:15:51.150719+0000","last_undegraded":"2024-02-13T14:15:51.150719+0000","last_fullsized":"2024-02-13T14:15:51.150719+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:31:26.104665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147897+0000","last_change":"2024-02-13T14:15:27.641831+0000","last_active":"2024-02-13T14:15:51.147897+0000","last_peered":"2024-02-13T14:15:51.147897+0000","last_clean":"2024-02-13T14:15:51.147897+0000","last_became_active":"2024-02-13T14:15:27.640612+0000","last_became_peered":"2024-02-13T14:15:27.640612+0000","last_unstale":"2024-02-13T14:15:51.147897+0000","last_undegraded":"2024-02-13T14:15:51.147897+0000","last_fullsized":"2024-02-13T14:15:51.147897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:20:21.915769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149702+0000","last_change":"2024-02-13T14:15:23.622834+0000","last_active":"2024-02-13T14:15:51.149702+0000","last_peered":"2024-02-13T14:15:51.149702+0000","last_clean":"2024-02-13T14:15:51.149702+0000","last_became_active":"2024-02-13T14:15:23.622511+0000","last_became_peered":"2024-02-13T14:15:23.622511+0000","last_unstale":"2024-02-13T14:15:51.149702+0000","last_undegraded":"2024-02-13T14:15:51.149702+0000","last_fullsized":"2024-02-13T14:15:51.149702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:45:17.706098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149161+0000","last_change":"2024-02-13T14:15:25.640119+0000","last_active":"2024-02-13T14:15:51.149161+0000","last_peered":"2024-02-13T14:15:51.149161+0000","last_clean":"2024-02-13T14:15:51.149161+0000","last_became_active":"2024-02-13T14:15:25.639549+0000","last_became_peered":"2024-02-13T14:15:25.639549+0000","last_unstale":"2024-02-13T14:15:51.149161+0000","last_undegraded":"2024-02-13T14:15:51.149161+0000","last_fullsized":"2024-02-13T14:15:51.149161+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:11:52.229637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146710+0000","last_change":"2024-02-13T14:15:29.650084+0000","last_active":"2024-02-13T14:15:51.146710+0000","last_peered":"2024-02-13T14:15:51.146710+0000","last_clean":"2024-02-13T14:15:51.146710+0000","last_became_active":"2024-02-13T14:15:29.648790+0000","last_became_peered":"2024-02-13T14:15:29.648790+0000","last_unstale":"2024-02-13T14:15:51.146710+0000","last_undegraded":"2024-02-13T14:15:51.146710+0000","last_fullsized":"2024-02-13T14:15:51.146710+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:51:33.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150077+0000","last_change":"2024-02-13T14:15:27.640984+0000","last_active":"2024-02-13T14:15:51.150077+0000","last_peered":"2024-02-13T14:15:51.150077+0000","last_clean":"2024-02-13T14:15:51.150077+0000","last_became_active":"2024-02-13T14:15:27.640600+0000","last_became_peered":"2024-02-13T14:15:27.640600+0000","last_unstale":"2024-02-13T14:15:51.150077+0000","last_undegraded":"2024-02-13T14:15:51.150077+0000","last_fullsized":"2024-02-13T14:15:51.150077+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:25:20.034995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149517+0000","last_change":"2024-02-13T14:15:23.623943+0000","last_active":"2024-02-13T14:15:51.149517+0000","last_peered":"2024-02-13T14:15:51.149517+0000","last_clean":"2024-02-13T14:15:51.149517+0000","last_became_active":"2024-02-13T14:15:23.623658+0000","last_became_peered":"2024-02-13T14:15:23.623658+0000","last_unstale":"2024-02-13T14:15:51.149517+0000","last_undegraded":"2024-02-13T14:15:51.149517+0000","last_fullsized":"2024-02-13T14:15:51.149517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:21:10.749864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149122+0000","last_change":"2024-02-13T14:15:25.639691+0000","last_active":"2024-02-13T14:15:51.149122+0000","last_peered":"2024-02-13T14:15:51.149122+0000","last_clean":"2024-02-13T14:15:51.149122+0000","last_became_active":"2024-02-13T14:15:25.639293+0000","last_became_peered":"2024-02-13T14:15:25.639293+0000","last_unstale":"2024-02-13T14:15:51.149122+0000","last_undegraded":"2024-02-13T14:15:51.149122+0000","last_fullsized":"2024-02-13T14:15:51.149122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:39:09.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148115+0000","last_change":"2024-02-13T14:15:29.645303+0000","last_active":"2024-02-13T14:15:51.148115+0000","last_peered":"2024-02-13T14:15:51.148115+0000","last_clean":"2024-02-13T14:15:51.148115+0000","last_became_active":"2024-02-13T14:15:29.644992+0000","last_became_peered":"2024-02-13T14:15:29.644992+0000","last_unstale":"2024-02-13T14:15:51.148115+0000","last_undegraded":"2024-02-13T14:15:51.148115+0000","last_fullsized":"2024-02-13T14:15:51.148115+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:55:36.492078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150112+0000","last_change":"2024-02-13T14:15:27.640101+0000","last_active":"2024-02-13T14:15:51.150112+0000","last_peered":"2024-02-13T14:15:51.150112+0000","last_clean":"2024-02-13T14:15:51.150112+0000","last_became_active":"2024-02-13T14:15:27.639712+0000","last_became_peered":"2024-02-13T14:15:27.639712+0000","last_unstale":"2024-02-13T14:15:51.150112+0000","last_undegraded":"2024-02-13T14:15:51.150112+0000","last_fullsized":"2024-02-13T14:15:51.150112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:49.267601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150547+0000","last_change":"2024-02-13T14:15:23.618714+0000","last_active":"2024-02-13T14:15:51.150547+0000","last_peered":"2024-02-13T14:15:51.150547+0000","last_clean":"2024-02-13T14:15:51.150547+0000","last_became_active":"2024-02-13T14:15:23.617832+0000","last_became_peered":"2024-02-13T14:15:23.617832+0000","last_unstale":"2024-02-13T14:15:51.150547+0000","last_undegraded":"2024-02-13T14:15:51.150547+0000","last_fullsized":"2024-02-13T14:15:51.150547+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:35:15.756097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151435+0000","last_change":"2024-02-13T14:15:25.632979+0000","last_active":"2024-02-13T14:15:51.151435+0000","last_peered":"2024-02-13T14:15:51.151435+0000","last_clean":"2024-02-13T14:15:51.151435+0000","last_became_active":"2024-02-13T14:15:25.632544+0000","last_became_peered":"2024-02-13T14:15:25.632544+0000","last_unstale":"2024-02-13T14:15:51.151435+0000","last_undegraded":"2024-02-13T14:15:51.151435+0000","last_fullsized":"2024-02-13T14:15:51.151435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:49:06.988964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149918+0000","last_change":"2024-02-13T14:15:29.646777+0000","last_active":"2024-02-13T14:15:51.149918+0000","last_peered":"2024-02-13T14:15:51.149918+0000","last_clean":"2024-02-13T14:15:51.149918+0000","last_became_active":"2024-02-13T14:15:29.646028+0000","last_became_peered":"2024-02-13T14:15:29.646028+0000","last_unstale":"2024-02-13T14:15:51.149918+0000","last_undegraded":"2024-02-13T14:15:51.149918+0000","last_fullsized":"2024-02-13T14:15:51.149918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:03:35.458796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150033+0000","last_change":"2024-02-13T14:15:27.640177+0000","last_active":"2024-02-13T14:15:51.150033+0000","last_peered":"2024-02-13T14:15:51.150033+0000","last_clean":"2024-02-13T14:15:51.150033+0000","last_became_active":"2024-02-13T14:15:27.639796+0000","last_became_peered":"2024-02-13T14:15:27.639796+0000","last_unstale":"2024-02-13T14:15:51.150033+0000","last_undegraded":"2024-02-13T14:15:51.150033+0000","last_fullsized":"2024-02-13T14:15:51.150033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:19:19.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150558+0000","last_change":"2024-02-13T14:15:23.617880+0000","last_active":"2024-02-13T14:15:51.150558+0000","last_peered":"2024-02-13T14:15:51.150558+0000","last_clean":"2024-02-13T14:15:51.150558+0000","last_became_active":"2024-02-13T14:15:23.617098+0000","last_became_peered":"2024-02-13T14:15:23.617098+0000","last_unstale":"2024-02-13T14:15:51.150558+0000","last_undegraded":"2024-02-13T14:15:51.150558+0000","last_fullsized":"2024-02-13T14:15:51.150558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:47:43.267755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150338+0000","last_change":"2024-02-13T14:15:25.632775+0000","last_active":"2024-02-13T14:15:51.150338+0000","last_peered":"2024-02-13T14:15:51.150338+0000","last_clean":"2024-02-13T14:15:51.150338+0000","last_became_active":"2024-02-13T14:15:25.632470+0000","last_became_peered":"2024-02-13T14:15:25.632470+0000","last_unstale":"2024-02-13T14:15:51.150338+0000","last_undegraded":"2024-02-13T14:15:51.150338+0000","last_fullsized":"2024-02-13T14:15:51.150338+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:19:18.279751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150858+0000","last_change":"2024-02-13T14:15:29.647903+0000","last_active":"2024-02-13T14:15:51.150858+0000","last_peered":"2024-02-13T14:15:51.150858+0000","last_clean":"2024-02-13T14:15:51.150858+0000","last_became_active":"2024-02-13T14:15:29.647003+0000","last_became_peered":"2024-02-13T14:15:29.647003+0000","last_unstale":"2024-02-13T14:15:51.150858+0000","last_undegraded":"2024-02-13T14:15:51.150858+0000","last_fullsized":"2024-02-13T14:15:51.150858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:05:46.480579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149686+0000","last_change":"2024-02-13T14:15:27.641240+0000","last_active":"2024-02-13T14:15:51.149686+0000","last_peered":"2024-02-13T14:15:51.149686+0000","last_clean":"2024-02-13T14:15:51.149686+0000","last_became_active":"2024-02-13T14:15:27.640912+0000","last_became_peered":"2024-02-13T14:15:27.640912+0000","last_unstale":"2024-02-13T14:15:51.149686+0000","last_undegraded":"2024-02-13T14:15:51.149686+0000","last_fullsized":"2024-02-13T14:15:51.149686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:36:27.153329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149324+0000","last_change":"2024-02-13T14:15:23.619731+0000","last_active":"2024-02-13T14:15:51.149324+0000","last_peered":"2024-02-13T14:15:51.149324+0000","last_clean":"2024-02-13T14:15:51.149324+0000","last_became_active":"2024-02-13T14:15:23.618669+0000","last_became_peered":"2024-02-13T14:15:23.618669+0000","last_unstale":"2024-02-13T14:15:51.149324+0000","last_undegraded":"2024-02-13T14:15:51.149324+0000","last_fullsized":"2024-02-13T14:15:51.149324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:01:40.159041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148929+0000","last_change":"2024-02-13T14:15:25.632087+0000","last_active":"2024-02-13T14:15:51.148929+0000","last_peered":"2024-02-13T14:15:51.148929+0000","last_clean":"2024-02-13T14:15:51.148929+0000","last_became_active":"2024-02-13T14:15:25.631709+0000","last_became_peered":"2024-02-13T14:15:25.631709+0000","last_unstale":"2024-02-13T14:15:51.148929+0000","last_undegraded":"2024-02-13T14:15:51.148929+0000","last_fullsized":"2024-02-13T14:15:51.148929+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:54:53.929894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148469+0000","last_change":"2024-02-13T14:15:29.650817+0000","last_active":"2024-02-13T14:15:51.148469+0000","last_peered":"2024-02-13T14:15:51.148469+0000","last_clean":"2024-02-13T14:15:51.148469+0000","last_became_active":"2024-02-13T14:15:29.650352+0000","last_became_peered":"2024-02-13T14:15:29.650352+0000","last_unstale":"2024-02-13T14:15:51.148469+0000","last_undegraded":"2024-02-13T14:15:51.148469+0000","last_fullsized":"2024-02-13T14:15:51.148469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:07.103058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150352+0000","last_change":"2024-02-13T14:15:27.641910+0000","last_active":"2024-02-13T14:15:51.150352+0000","last_peered":"2024-02-13T14:15:51.150352+0000","last_clean":"2024-02-13T14:15:51.150352+0000","last_became_active":"2024-02-13T14:15:27.641619+0000","last_became_peered":"2024-02-13T14:15:27.641619+0000","last_unstale":"2024-02-13T14:15:51.150352+0000","last_undegraded":"2024-02-13T14:15:51.150352+0000","last_fullsized":"2024-02-13T14:15:51.150352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:50:21.787723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151955+0000","last_change":"2024-02-13T14:15:23.619919+0000","last_active":"2024-02-13T14:15:51.151955+0000","last_peered":"2024-02-13T14:15:51.151955+0000","last_clean":"2024-02-13T14:15:51.151955+0000","last_became_active":"2024-02-13T14:15:23.619361+0000","last_became_peered":"2024-02-13T14:15:23.619361+0000","last_unstale":"2024-02-13T14:15:51.151955+0000","last_undegraded":"2024-02-13T14:15:51.151955+0000","last_fullsized":"2024-02-13T14:15:51.151955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:47:48.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149003+0000","last_change":"2024-02-13T14:15:25.631457+0000","last_active":"2024-02-13T14:15:51.149003+0000","last_peered":"2024-02-13T14:15:51.149003+0000","last_clean":"2024-02-13T14:15:51.149003+0000","last_became_active":"2024-02-13T14:15:25.631063+0000","last_became_peered":"2024-02-13T14:15:25.631063+0000","last_unstale":"2024-02-13T14:15:51.149003+0000","last_undegraded":"2024-02-13T14:15:51.149003+0000","last_fullsized":"2024-02-13T14:15:51.149003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:45:55.797820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149338+0000","last_change":"2024-02-13T14:15:29.648463+0000","last_active":"2024-02-13T14:15:51.149338+0000","last_peered":"2024-02-13T14:15:51.149338+0000","last_clean":"2024-02-13T14:15:51.149338+0000","last_became_active":"2024-02-13T14:15:29.647919+0000","last_became_peered":"2024-02-13T14:15:29.647919+0000","last_unstale":"2024-02-13T14:15:51.149338+0000","last_undegraded":"2024-02-13T14:15:51.149338+0000","last_fullsized":"2024-02-13T14:15:51.149338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:02:23.090634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668665+0000","last_change":"2024-02-13T14:15:27.643210+0000","last_active":"2024-02-13T14:15:58.668665+0000","last_peered":"2024-02-13T14:15:58.668665+0000","last_clean":"2024-02-13T14:15:58.668665+0000","last_became_active":"2024-02-13T14:15:27.642379+0000","last_became_peered":"2024-02-13T14:15:27.642379+0000","last_unstale":"2024-02-13T14:15:58.668665+0000","last_undegraded":"2024-02-13T14:15:58.668665+0000","last_fullsized":"2024-02-13T14:15:58.668665+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:46:12.920991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146895+0000","last_change":"2024-02-13T14:15:23.622973+0000","last_active":"2024-02-13T14:15:51.146895+0000","last_peered":"2024-02-13T14:15:51.146895+0000","last_clean":"2024-02-13T14:15:51.146895+0000","last_became_active":"2024-02-13T14:15:23.622680+0000","last_became_peered":"2024-02-13T14:15:23.622680+0000","last_unstale":"2024-02-13T14:15:51.146895+0000","last_undegraded":"2024-02-13T14:15:51.146895+0000","last_fullsized":"2024-02-13T14:15:51.146895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:42:39.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146083+0000","last_change":"2024-02-13T14:15:25.638424+0000","last_active":"2024-02-13T14:15:51.146083+0000","last_peered":"2024-02-13T14:15:51.146083+0000","last_clean":"2024-02-13T14:15:51.146083+0000","last_became_active":"2024-02-13T14:15:25.638015+0000","last_became_peered":"2024-02-13T14:15:25.638015+0000","last_unstale":"2024-02-13T14:15:51.146083+0000","last_undegraded":"2024-02-13T14:15:51.146083+0000","last_fullsized":"2024-02-13T14:15:51.146083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:35:10.188869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151238+0000","last_change":"2024-02-13T14:15:29.648091+0000","last_active":"2024-02-13T14:15:51.151238+0000","last_peered":"2024-02-13T14:15:51.151238+0000","last_clean":"2024-02-13T14:15:51.151238+0000","last_became_active":"2024-02-13T14:15:29.647683+0000","last_became_peered":"2024-02-13T14:15:29.647683+0000","last_unstale":"2024-02-13T14:15:51.151238+0000","last_undegraded":"2024-02-13T14:15:51.151238+0000","last_fullsized":"2024-02-13T14:15:51.151238+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:54:06.592494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148769+0000","last_change":"2024-02-13T14:15:27.643425+0000","last_active":"2024-02-13T14:15:51.148769+0000","last_peered":"2024-02-13T14:15:51.148769+0000","last_clean":"2024-02-13T14:15:51.148769+0000","last_became_active":"2024-02-13T14:15:27.643068+0000","last_became_peered":"2024-02-13T14:15:27.643068+0000","last_unstale":"2024-02-13T14:15:51.148769+0000","last_undegraded":"2024-02-13T14:15:51.148769+0000","last_fullsized":"2024-02-13T14:15:51.148769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:32:14.274046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148026+0000","last_change":"2024-02-13T14:15:23.620818+0000","last_active":"2024-02-13T14:15:51.148026+0000","last_peered":"2024-02-13T14:15:51.148026+0000","last_clean":"2024-02-13T14:15:51.148026+0000","last_became_active":"2024-02-13T14:15:23.620520+0000","last_became_peered":"2024-02-13T14:15:23.620520+0000","last_unstale":"2024-02-13T14:15:51.148026+0000","last_undegraded":"2024-02-13T14:15:51.148026+0000","last_fullsized":"2024-02-13T14:15:51.148026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:18.701338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146464+0000","last_change":"2024-02-13T14:15:25.639838+0000","last_active":"2024-02-13T14:15:51.146464+0000","last_peered":"2024-02-13T14:15:51.146464+0000","last_clean":"2024-02-13T14:15:51.146464+0000","last_became_active":"2024-02-13T14:15:25.639119+0000","last_became_peered":"2024-02-13T14:15:25.639119+0000","last_unstale":"2024-02-13T14:15:51.146464+0000","last_undegraded":"2024-02-13T14:15:51.146464+0000","last_fullsized":"2024-02-13T14:15:51.146464+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:59:13.268372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151147+0000","last_change":"2024-02-13T14:15:29.648555+0000","last_active":"2024-02-13T14:15:51.151147+0000","last_peered":"2024-02-13T14:15:51.151147+0000","last_clean":"2024-02-13T14:15:51.151147+0000","last_became_active":"2024-02-13T14:15:29.647901+0000","last_became_peered":"2024-02-13T14:15:29.647901+0000","last_unstale":"2024-02-13T14:15:51.151147+0000","last_undegraded":"2024-02-13T14:15:51.151147+0000","last_fullsized":"2024-02-13T14:15:51.151147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:08:58.959202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146147+0000","last_change":"2024-02-13T14:15:27.645059+0000","last_active":"2024-02-13T14:15:51.146147+0000","last_peered":"2024-02-13T14:15:51.146147+0000","last_clean":"2024-02-13T14:15:51.146147+0000","last_became_active":"2024-02-13T14:15:27.644421+0000","last_became_peered":"2024-02-13T14:15:27.644421+0000","last_unstale":"2024-02-13T14:15:51.146147+0000","last_undegraded":"2024-02-13T14:15:51.146147+0000","last_fullsized":"2024-02-13T14:15:51.146147+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:03:59.473260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152081+0000","last_change":"2024-02-13T14:15:23.620877+0000","last_active":"2024-02-13T14:15:51.152081+0000","last_peered":"2024-02-13T14:15:51.152081+0000","last_clean":"2024-02-13T14:15:51.152081+0000","last_became_active":"2024-02-13T14:15:23.620356+0000","last_became_peered":"2024-02-13T14:15:23.620356+0000","last_unstale":"2024-02-13T14:15:51.152081+0000","last_undegraded":"2024-02-13T14:15:51.152081+0000","last_fullsized":"2024-02-13T14:15:51.152081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:33.686661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151256+0000","last_change":"2024-02-13T14:15:25.636137+0000","last_active":"2024-02-13T14:15:51.151256+0000","last_peered":"2024-02-13T14:15:51.151256+0000","last_clean":"2024-02-13T14:15:51.151256+0000","last_became_active":"2024-02-13T14:15:25.635594+0000","last_became_peered":"2024-02-13T14:15:25.635594+0000","last_unstale":"2024-02-13T14:15:51.151256+0000","last_undegraded":"2024-02-13T14:15:51.151256+0000","last_fullsized":"2024-02-13T14:15:51.151256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:41.070936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150310+0000","last_change":"2024-02-13T14:15:29.645215+0000","last_active":"2024-02-13T14:15:51.150310+0000","last_peered":"2024-02-13T14:15:51.150310+0000","last_clean":"2024-02-13T14:15:51.150310+0000","last_became_active":"2024-02-13T14:15:29.644869+0000","last_became_peered":"2024-02-13T14:15:29.644869+0000","last_unstale":"2024-02-13T14:15:51.150310+0000","last_undegraded":"2024-02-13T14:15:51.150310+0000","last_fullsized":"2024-02-13T14:15:51.150310+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:49:49.161277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.145938+0000","last_change":"2024-02-13T14:15:27.643536+0000","last_active":"2024-02-13T14:15:51.145938+0000","last_peered":"2024-02-13T14:15:51.145938+0000","last_clean":"2024-02-13T14:15:51.145938+0000","last_became_active":"2024-02-13T14:15:27.643173+0000","last_became_peered":"2024-02-13T14:15:27.643173+0000","last_unstale":"2024-02-13T14:15:51.145938+0000","last_undegraded":"2024-02-13T14:15:51.145938+0000","last_fullsized":"2024-02-13T14:15:51.145938+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:04:45.289058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150925+0000","last_change":"2024-02-13T14:15:23.621820+0000","last_active":"2024-02-13T14:15:51.150925+0000","last_peered":"2024-02-13T14:15:51.150925+0000","last_clean":"2024-02-13T14:15:51.150925+0000","last_became_active":"2024-02-13T14:15:23.621457+0000","last_became_peered":"2024-02-13T14:15:23.621457+0000","last_unstale":"2024-02-13T14:15:51.150925+0000","last_undegraded":"2024-02-13T14:15:51.150925+0000","last_fullsized":"2024-02-13T14:15:51.150925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:03:21.989986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149384+0000","last_change":"2024-02-13T14:15:25.640447+0000","last_active":"2024-02-13T14:15:51.149384+0000","last_peered":"2024-02-13T14:15:51.149384+0000","last_clean":"2024-02-13T14:15:51.149384+0000","last_became_active":"2024-02-13T14:15:25.638041+0000","last_became_peered":"2024-02-13T14:15:25.638041+0000","last_unstale":"2024-02-13T14:15:51.149384+0000","last_undegraded":"2024-02-13T14:15:51.149384+0000","last_fullsized":"2024-02-13T14:15:51.149384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:43:48.962133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148337+0000","last_change":"2024-02-13T14:15:29.644948+0000","last_active":"2024-02-13T14:15:51.148337+0000","last_peered":"2024-02-13T14:15:51.148337+0000","last_clean":"2024-02-13T14:15:51.148337+0000","last_became_active":"2024-02-13T14:15:29.644522+0000","last_became_peered":"2024-02-13T14:15:29.644522+0000","last_unstale":"2024-02-13T14:15:51.148337+0000","last_undegraded":"2024-02-13T14:15:51.148337+0000","last_fullsized":"2024-02-13T14:15:51.148337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:58:47.616179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150285+0000","last_change":"2024-02-13T14:15:27.640695+0000","last_active":"2024-02-13T14:15:51.150285+0000","last_peered":"2024-02-13T14:15:51.150285+0000","last_clean":"2024-02-13T14:15:51.150285+0000","last_became_active":"2024-02-13T14:15:27.640342+0000","last_became_peered":"2024-02-13T14:15:27.640342+0000","last_unstale":"2024-02-13T14:15:51.150285+0000","last_undegraded":"2024-02-13T14:15:51.150285+0000","last_fullsized":"2024-02-13T14:15:51.150285+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:24:42.246645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150236+0000","last_change":"2024-02-13T14:15:23.620111+0000","last_active":"2024-02-13T14:15:51.150236+0000","last_peered":"2024-02-13T14:15:51.150236+0000","last_clean":"2024-02-13T14:15:51.150236+0000","last_became_active":"2024-02-13T14:15:23.618976+0000","last_became_peered":"2024-02-13T14:15:23.618976+0000","last_unstale":"2024-02-13T14:15:51.150236+0000","last_undegraded":"2024-02-13T14:15:51.150236+0000","last_fullsized":"2024-02-13T14:15:51.150236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:38:48.463427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151773+0000","last_change":"2024-02-13T14:15:25.634282+0000","last_active":"2024-02-13T14:15:51.151773+0000","last_peered":"2024-02-13T14:15:51.151773+0000","last_clean":"2024-02-13T14:15:51.151773+0000","last_became_active":"2024-02-13T14:15:25.633915+0000","last_became_peered":"2024-02-13T14:15:25.633915+0000","last_unstale":"2024-02-13T14:15:51.151773+0000","last_undegraded":"2024-02-13T14:15:51.151773+0000","last_fullsized":"2024-02-13T14:15:51.151773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:29:51.823704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148330+0000","last_change":"2024-02-13T14:15:29.644167+0000","last_active":"2024-02-13T14:15:51.148330+0000","last_peered":"2024-02-13T14:15:51.148330+0000","last_clean":"2024-02-13T14:15:51.148330+0000","last_became_active":"2024-02-13T14:15:29.643846+0000","last_became_peered":"2024-02-13T14:15:29.643846+0000","last_unstale":"2024-02-13T14:15:51.148330+0000","last_undegraded":"2024-02-13T14:15:51.148330+0000","last_fullsized":"2024-02-13T14:15:51.148330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:14:08.875512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146242+0000","last_change":"2024-02-13T14:15:27.642965+0000","last_active":"2024-02-13T14:15:51.146242+0000","last_peered":"2024-02-13T14:15:51.146242+0000","last_clean":"2024-02-13T14:15:51.146242+0000","last_became_active":"2024-02-13T14:15:27.642585+0000","last_became_peered":"2024-02-13T14:15:27.642585+0000","last_unstale":"2024-02-13T14:15:51.146242+0000","last_undegraded":"2024-02-13T14:15:51.146242+0000","last_fullsized":"2024-02-13T14:15:51.146242+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:48:25.621534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151204+0000","last_change":"2024-02-13T14:15:23.620750+0000","last_active":"2024-02-13T14:15:51.151204+0000","last_peered":"2024-02-13T14:15:51.151204+0000","last_clean":"2024-02-13T14:15:51.151204+0000","last_became_active":"2024-02-13T14:15:23.620347+0000","last_became_peered":"2024-02-13T14:15:23.620347+0000","last_unstale":"2024-02-13T14:15:51.151204+0000","last_undegraded":"2024-02-13T14:15:51.151204+0000","last_fullsized":"2024-02-13T14:15:51.151204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:06:33.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150069+0000","last_change":"2024-02-13T14:15:25.632939+0000","last_active":"2024-02-13T14:15:51.150069+0000","last_peered":"2024-02-13T14:15:51.150069+0000","last_clean":"2024-02-13T14:15:51.150069+0000","last_became_active":"2024-02-13T14:15:25.632560+0000","last_became_peered":"2024-02-13T14:15:25.632560+0000","last_unstale":"2024-02-13T14:15:51.150069+0000","last_undegraded":"2024-02-13T14:15:51.150069+0000","last_fullsized":"2024-02-13T14:15:51.150069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:51:13.680314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147259+0000","last_change":"2024-02-13T14:15:29.649650+0000","last_active":"2024-02-13T14:15:51.147259+0000","last_peered":"2024-02-13T14:15:51.147259+0000","last_clean":"2024-02-13T14:15:51.147259+0000","last_became_active":"2024-02-13T14:15:29.649249+0000","last_became_peered":"2024-02-13T14:15:29.649249+0000","last_unstale":"2024-02-13T14:15:51.147259+0000","last_undegraded":"2024-02-13T14:15:51.147259+0000","last_fullsized":"2024-02-13T14:15:51.147259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:48:39.046537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668685+0000","last_change":"2024-02-13T14:15:27.643637+0000","last_active":"2024-02-13T14:15:58.668685+0000","last_peered":"2024-02-13T14:15:58.668685+0000","last_clean":"2024-02-13T14:15:58.668685+0000","last_became_active":"2024-02-13T14:15:27.643224+0000","last_became_peered":"2024-02-13T14:15:27.643224+0000","last_unstale":"2024-02-13T14:15:58.668685+0000","last_undegraded":"2024-02-13T14:15:58.668685+0000","last_fullsized":"2024-02-13T14:15:58.668685+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:33:48.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149508+0000","last_change":"2024-02-13T14:15:23.618907+0000","last_active":"2024-02-13T14:15:51.149508+0000","last_peered":"2024-02-13T14:15:51.149508+0000","last_clean":"2024-02-13T14:15:51.149508+0000","last_became_active":"2024-02-13T14:15:23.617121+0000","last_became_peered":"2024-02-13T14:15:23.617121+0000","last_unstale":"2024-02-13T14:15:51.149508+0000","last_undegraded":"2024-02-13T14:15:51.149508+0000","last_fullsized":"2024-02-13T14:15:51.149508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:44:58.554123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146427+0000","last_change":"2024-02-13T14:15:25.642435+0000","last_active":"2024-02-13T14:15:51.146427+0000","last_peered":"2024-02-13T14:15:51.146427+0000","last_clean":"2024-02-13T14:15:51.146427+0000","last_became_active":"2024-02-13T14:15:25.642078+0000","last_became_peered":"2024-02-13T14:15:25.642078+0000","last_unstale":"2024-02-13T14:15:51.146427+0000","last_undegraded":"2024-02-13T14:15:51.146427+0000","last_fullsized":"2024-02-13T14:15:51.146427+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:45:06.913112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151566+0000","last_change":"2024-02-13T14:15:29.647394+0000","last_active":"2024-02-13T14:15:51.151566+0000","last_peered":"2024-02-13T14:15:51.151566+0000","last_clean":"2024-02-13T14:15:51.151566+0000","last_became_active":"2024-02-13T14:15:29.646158+0000","last_became_peered":"2024-02-13T14:15:29.646158+0000","last_unstale":"2024-02-13T14:15:51.151566+0000","last_undegraded":"2024-02-13T14:15:51.151566+0000","last_fullsized":"2024-02-13T14:15:51.151566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:54:51.741665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668642+0000","last_change":"2024-02-13T14:15:27.641066+0000","last_active":"2024-02-13T14:15:58.668642+0000","last_peered":"2024-02-13T14:15:58.668642+0000","last_clean":"2024-02-13T14:15:58.668642+0000","last_became_active":"2024-02-13T14:15:27.640758+0000","last_became_peered":"2024-02-13T14:15:27.640758+0000","last_unstale":"2024-02-13T14:15:58.668642+0000","last_undegraded":"2024-02-13T14:15:58.668642+0000","last_fullsized":"2024-02-13T14:15:58.668642+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:21:49.097191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148618+0000","last_change":"2024-02-13T14:15:23.618464+0000","last_active":"2024-02-13T14:15:51.148618+0000","last_peered":"2024-02-13T14:15:51.148618+0000","last_clean":"2024-02-13T14:15:51.148618+0000","last_became_active":"2024-02-13T14:15:23.617343+0000","last_became_peered":"2024-02-13T14:15:23.617343+0000","last_unstale":"2024-02-13T14:15:51.148618+0000","last_undegraded":"2024-02-13T14:15:51.148618+0000","last_fullsized":"2024-02-13T14:15:51.148618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:41:01.716785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150656+0000","last_change":"2024-02-13T14:15:25.633331+0000","last_active":"2024-02-13T14:15:51.150656+0000","last_peered":"2024-02-13T14:15:51.150656+0000","last_clean":"2024-02-13T14:15:51.150656+0000","last_became_active":"2024-02-13T14:15:25.632968+0000","last_became_peered":"2024-02-13T14:15:25.632968+0000","last_unstale":"2024-02-13T14:15:51.150656+0000","last_undegraded":"2024-02-13T14:15:51.150656+0000","last_fullsized":"2024-02-13T14:15:51.150656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:41:14.441961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150373+0000","last_change":"2024-02-13T14:15:29.647789+0000","last_active":"2024-02-13T14:15:51.150373+0000","last_peered":"2024-02-13T14:15:51.150373+0000","last_clean":"2024-02-13T14:15:51.150373+0000","last_became_active":"2024-02-13T14:15:29.647388+0000","last_became_peered":"2024-02-13T14:15:29.647388+0000","last_unstale":"2024-02-13T14:15:51.150373+0000","last_undegraded":"2024-02-13T14:15:51.150373+0000","last_fullsized":"2024-02-13T14:15:51.150373+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:58:44.557944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149021+0000","last_change":"2024-02-13T14:15:27.645782+0000","last_active":"2024-02-13T14:15:51.149021+0000","last_peered":"2024-02-13T14:15:51.149021+0000","last_clean":"2024-02-13T14:15:51.149021+0000","last_became_active":"2024-02-13T14:15:27.644997+0000","last_became_peered":"2024-02-13T14:15:27.644997+0000","last_unstale":"2024-02-13T14:15:51.149021+0000","last_undegraded":"2024-02-13T14:15:51.149021+0000","last_fullsized":"2024-02-13T14:15:51.149021+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:02:50.332899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150739+0000","last_change":"2024-02-13T14:15:23.620190+0000","last_active":"2024-02-13T14:15:51.150739+0000","last_peered":"2024-02-13T14:15:51.150739+0000","last_clean":"2024-02-13T14:15:51.150739+0000","last_became_active":"2024-02-13T14:15:23.619620+0000","last_became_peered":"2024-02-13T14:15:23.619620+0000","last_unstale":"2024-02-13T14:15:51.150739+0000","last_undegraded":"2024-02-13T14:15:51.150739+0000","last_fullsized":"2024-02-13T14:15:51.150739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:44:41.067619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149853+0000","last_change":"2024-02-13T14:15:25.636616+0000","last_active":"2024-02-13T14:15:51.149853+0000","last_peered":"2024-02-13T14:15:51.149853+0000","last_clean":"2024-02-13T14:15:51.149853+0000","last_became_active":"2024-02-13T14:15:25.635951+0000","last_became_peered":"2024-02-13T14:15:25.635951+0000","last_unstale":"2024-02-13T14:15:51.149853+0000","last_undegraded":"2024-02-13T14:15:51.149853+0000","last_fullsized":"2024-02-13T14:15:51.149853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:03:25.987871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148791+0000","last_change":"2024-02-13T14:15:29.648152+0000","last_active":"2024-02-13T14:15:51.148791+0000","last_peered":"2024-02-13T14:15:51.148791+0000","last_clean":"2024-02-13T14:15:51.148791+0000","last_became_active":"2024-02-13T14:15:29.647770+0000","last_became_peered":"2024-02-13T14:15:29.647770+0000","last_unstale":"2024-02-13T14:15:51.148791+0000","last_undegraded":"2024-02-13T14:15:51.148791+0000","last_fullsized":"2024-02-13T14:15:51.148791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:44:47.872956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148877+0000","last_change":"2024-02-13T14:15:27.646752+0000","last_active":"2024-02-13T14:15:51.148877+0000","last_peered":"2024-02-13T14:15:51.148877+0000","last_clean":"2024-02-13T14:15:51.148877+0000","last_became_active":"2024-02-13T14:15:27.646401+0000","last_became_peered":"2024-02-13T14:15:27.646401+0000","last_unstale":"2024-02-13T14:15:51.148877+0000","last_undegraded":"2024-02-13T14:15:51.148877+0000","last_fullsized":"2024-02-13T14:15:51.148877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:21:52.378118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148670+0000","last_change":"2024-02-13T14:15:23.621049+0000","last_active":"2024-02-13T14:15:51.148670+0000","last_peered":"2024-02-13T14:15:51.148670+0000","last_clean":"2024-02-13T14:15:51.148670+0000","last_became_active":"2024-02-13T14:15:23.620705+0000","last_became_peered":"2024-02-13T14:15:23.620705+0000","last_unstale":"2024-02-13T14:15:51.148670+0000","last_undegraded":"2024-02-13T14:15:51.148670+0000","last_fullsized":"2024-02-13T14:15:51.148670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:50:45.636400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149054+0000","last_change":"2024-02-13T14:15:25.631891+0000","last_active":"2024-02-13T14:15:51.149054+0000","last_peered":"2024-02-13T14:15:51.149054+0000","last_clean":"2024-02-13T14:15:51.149054+0000","last_became_active":"2024-02-13T14:15:25.631483+0000","last_became_peered":"2024-02-13T14:15:25.631483+0000","last_unstale":"2024-02-13T14:15:51.149054+0000","last_undegraded":"2024-02-13T14:15:51.149054+0000","last_fullsized":"2024-02-13T14:15:51.149054+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:25:10.383502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149120+0000","last_change":"2024-02-13T14:15:29.644727+0000","last_active":"2024-02-13T14:15:51.149120+0000","last_peered":"2024-02-13T14:15:51.149120+0000","last_clean":"2024-02-13T14:15:51.149120+0000","last_became_active":"2024-02-13T14:15:29.644457+0000","last_became_peered":"2024-02-13T14:15:29.644457+0000","last_unstale":"2024-02-13T14:15:51.149120+0000","last_undegraded":"2024-02-13T14:15:51.149120+0000","last_fullsized":"2024-02-13T14:15:51.149120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:35:16.730471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150437+0000","last_change":"2024-02-13T14:15:23.616989+0000","last_active":"2024-02-13T14:15:51.150437+0000","last_peered":"2024-02-13T14:15:51.150437+0000","last_clean":"2024-02-13T14:15:51.150437+0000","last_became_active":"2024-02-13T14:15:23.616123+0000","last_became_peered":"2024-02-13T14:15:23.616123+0000","last_unstale":"2024-02-13T14:15:51.150437+0000","last_undegraded":"2024-02-13T14:15:51.150437+0000","last_fullsized":"2024-02-13T14:15:51.150437+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:18:04.651463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150047+0000","last_change":"2024-02-13T14:15:25.632523+0000","last_active":"2024-02-13T14:15:51.150047+0000","last_peered":"2024-02-13T14:15:51.150047+0000","last_clean":"2024-02-13T14:15:51.150047+0000","last_became_active":"2024-02-13T14:15:25.632205+0000","last_became_peered":"2024-02-13T14:15:25.632205+0000","last_unstale":"2024-02-13T14:15:51.150047+0000","last_undegraded":"2024-02-13T14:15:51.150047+0000","last_fullsized":"2024-02-13T14:15:51.150047+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:35:50.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668669+0000","last_change":"2024-02-13T14:15:27.641537+0000","last_active":"2024-02-13T14:15:58.668669+0000","last_peered":"2024-02-13T14:15:58.668669+0000","last_clean":"2024-02-13T14:15:58.668669+0000","last_became_active":"2024-02-13T14:15:27.640195+0000","last_became_peered":"2024-02-13T14:15:27.640195+0000","last_unstale":"2024-02-13T14:15:58.668669+0000","last_undegraded":"2024-02-13T14:15:58.668669+0000","last_fullsized":"2024-02-13T14:15:58.668669+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:36:53.918275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146573+0000","last_change":"2024-02-13T14:15:29.648143+0000","last_active":"2024-02-13T14:15:51.146573+0000","last_peered":"2024-02-13T14:15:51.146573+0000","last_clean":"2024-02-13T14:15:51.146573+0000","last_became_active":"2024-02-13T14:15:29.647699+0000","last_became_peered":"2024-02-13T14:15:29.647699+0000","last_unstale":"2024-02-13T14:15:51.146573+0000","last_undegraded":"2024-02-13T14:15:51.146573+0000","last_fullsized":"2024-02-13T14:15:51.146573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:59:39.522434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148284+0000","last_change":"2024-02-13T14:15:23.621266+0000","last_active":"2024-02-13T14:15:51.148284+0000","last_peered":"2024-02-13T14:15:51.148284+0000","last_clean":"2024-02-13T14:15:51.148284+0000","last_became_active":"2024-02-13T14:15:23.620838+0000","last_became_peered":"2024-02-13T14:15:23.620838+0000","last_unstale":"2024-02-13T14:15:51.148284+0000","last_undegraded":"2024-02-13T14:15:51.148284+0000","last_fullsized":"2024-02-13T14:15:51.148284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:11:15.677303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668464+0000","last_change":"2024-02-13T14:15:25.635009+0000","last_active":"2024-02-13T14:15:58.668464+0000","last_peered":"2024-02-13T14:15:58.668464+0000","last_clean":"2024-02-13T14:15:58.668464+0000","last_became_active":"2024-02-13T14:15:25.634395+0000","last_became_peered":"2024-02-13T14:15:25.634395+0000","last_unstale":"2024-02-13T14:15:58.668464+0000","last_undegraded":"2024-02-13T14:15:58.668464+0000","last_fullsized":"2024-02-13T14:15:58.668464+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:13:19.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148430+0000","last_change":"2024-02-13T14:15:27.643995+0000","last_active":"2024-02-13T14:15:51.148430+0000","last_peered":"2024-02-13T14:15:51.148430+0000","last_clean":"2024-02-13T14:15:51.148430+0000","last_became_active":"2024-02-13T14:15:27.640166+0000","last_became_peered":"2024-02-13T14:15:27.640166+0000","last_unstale":"2024-02-13T14:15:51.148430+0000","last_undegraded":"2024-02-13T14:15:51.148430+0000","last_fullsized":"2024-02-13T14:15:51.148430+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:07:32.813689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147845+0000","last_change":"2024-02-13T14:15:29.650567+0000","last_active":"2024-02-13T14:15:51.147845+0000","last_peered":"2024-02-13T14:15:51.147845+0000","last_clean":"2024-02-13T14:15:51.147845+0000","last_became_active":"2024-02-13T14:15:29.650245+0000","last_became_peered":"2024-02-13T14:15:29.650245+0000","last_unstale":"2024-02-13T14:15:51.147845+0000","last_undegraded":"2024-02-13T14:15:51.147845+0000","last_fullsized":"2024-02-13T14:15:51.147845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:07:49.909687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152102+0000","last_change":"2024-02-13T14:15:23.622382+0000","last_active":"2024-02-13T14:15:51.152102+0000","last_peered":"2024-02-13T14:15:51.152102+0000","last_clean":"2024-02-13T14:15:51.152102+0000","last_became_active":"2024-02-13T14:15:23.621821+0000","last_became_peered":"2024-02-13T14:15:23.621821+0000","last_unstale":"2024-02-13T14:15:51.152102+0000","last_undegraded":"2024-02-13T14:15:51.152102+0000","last_fullsized":"2024-02-13T14:15:51.152102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:34:21.313022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151323+0000","last_change":"2024-02-13T14:15:25.634245+0000","last_active":"2024-02-13T14:15:51.151323+0000","last_peered":"2024-02-13T14:15:51.151323+0000","last_clean":"2024-02-13T14:15:51.151323+0000","last_became_active":"2024-02-13T14:15:25.633731+0000","last_became_peered":"2024-02-13T14:15:25.633731+0000","last_unstale":"2024-02-13T14:15:51.151323+0000","last_undegraded":"2024-02-13T14:15:51.151323+0000","last_fullsized":"2024-02-13T14:15:51.151323+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:34:29.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147275+0000","last_change":"2024-02-13T14:15:27.644132+0000","last_active":"2024-02-13T14:15:51.147275+0000","last_peered":"2024-02-13T14:15:51.147275+0000","last_clean":"2024-02-13T14:15:51.147275+0000","last_became_active":"2024-02-13T14:15:27.643755+0000","last_became_peered":"2024-02-13T14:15:27.643755+0000","last_unstale":"2024-02-13T14:15:51.147275+0000","last_undegraded":"2024-02-13T14:15:51.147275+0000","last_fullsized":"2024-02-13T14:15:51.147275+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:51.227120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149521+0000","last_change":"2024-02-13T14:15:29.649490+0000","last_active":"2024-02-13T14:15:51.149521+0000","last_peered":"2024-02-13T14:15:51.149521+0000","last_clean":"2024-02-13T14:15:51.149521+0000","last_became_active":"2024-02-13T14:15:29.649110+0000","last_became_peered":"2024-02-13T14:15:29.649110+0000","last_unstale":"2024-02-13T14:15:51.149521+0000","last_undegraded":"2024-02-13T14:15:51.149521+0000","last_fullsized":"2024-02-13T14:15:51.149521+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:41:41.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147761+0000","last_change":"2024-02-13T14:15:25.632472+0000","last_active":"2024-02-13T14:15:51.147761+0000","last_peered":"2024-02-13T14:15:51.147761+0000","last_clean":"2024-02-13T14:15:51.147761+0000","last_became_active":"2024-02-13T14:15:25.632178+0000","last_became_peered":"2024-02-13T14:15:25.632178+0000","last_unstale":"2024-02-13T14:15:51.147761+0000","last_undegraded":"2024-02-13T14:15:51.147761+0000","last_fullsized":"2024-02-13T14:15:51.147761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:37:54.057582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146657+0000","last_change":"2024-02-13T14:15:23.620438+0000","last_active":"2024-02-13T14:15:51.146657+0000","last_peered":"2024-02-13T14:15:51.146657+0000","last_clean":"2024-02-13T14:15:51.146657+0000","last_became_active":"2024-02-13T14:15:23.620125+0000","last_became_peered":"2024-02-13T14:15:23.620125+0000","last_unstale":"2024-02-13T14:15:51.146657+0000","last_undegraded":"2024-02-13T14:15:51.146657+0000","last_fullsized":"2024-02-13T14:15:51.146657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:23:36.749528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149291+0000","last_change":"2024-02-13T14:15:27.645971+0000","last_active":"2024-02-13T14:15:51.149291+0000","last_peered":"2024-02-13T14:15:51.149291+0000","last_clean":"2024-02-13T14:15:51.149291+0000","last_became_active":"2024-02-13T14:15:27.645194+0000","last_became_peered":"2024-02-13T14:15:27.645194+0000","last_unstale":"2024-02-13T14:15:51.149291+0000","last_undegraded":"2024-02-13T14:15:51.149291+0000","last_fullsized":"2024-02-13T14:15:51.149291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:56:04.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149338+0000","last_change":"2024-02-13T14:15:29.643780+0000","last_active":"2024-02-13T14:15:51.149338+0000","last_peered":"2024-02-13T14:15:51.149338+0000","last_clean":"2024-02-13T14:15:51.149338+0000","last_became_active":"2024-02-13T14:15:29.643523+0000","last_became_peered":"2024-02-13T14:15:29.643523+0000","last_unstale":"2024-02-13T14:15:51.149338+0000","last_undegraded":"2024-02-13T14:15:51.149338+0000","last_fullsized":"2024-02-13T14:15:51.149338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:47:24.677905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146319+0000","last_change":"2024-02-13T14:15:25.642347+0000","last_active":"2024-02-13T14:15:51.146319+0000","last_peered":"2024-02-13T14:15:51.146319+0000","last_clean":"2024-02-13T14:15:51.146319+0000","last_became_active":"2024-02-13T14:15:25.642004+0000","last_became_peered":"2024-02-13T14:15:25.642004+0000","last_unstale":"2024-02-13T14:15:51.146319+0000","last_undegraded":"2024-02-13T14:15:51.146319+0000","last_fullsized":"2024-02-13T14:15:51.146319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:08:39.160948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150783+0000","last_change":"2024-02-13T14:15:23.619762+0000","last_active":"2024-02-13T14:15:51.150783+0000","last_peered":"2024-02-13T14:15:51.150783+0000","last_clean":"2024-02-13T14:15:51.150783+0000","last_became_active":"2024-02-13T14:15:23.619370+0000","last_became_peered":"2024-02-13T14:15:23.619370+0000","last_unstale":"2024-02-13T14:15:51.150783+0000","last_undegraded":"2024-02-13T14:15:51.150783+0000","last_fullsized":"2024-02-13T14:15:51.150783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:10:07.714428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.249303+0000","last_change":"2024-02-13T14:15:13.265703+0000","last_active":"2024-02-13T14:15:53.249303+0000","last_peered":"2024-02-13T14:15:53.249303+0000","last_clean":"2024-02-13T14:15:53.249303+0000","last_became_active":"2024-02-13T14:15:13.265341+0000","last_became_peered":"2024-02-13T14:15:13.265341+0000","last_unstale":"2024-02-13T14:15:53.249303+0000","last_undegraded":"2024-02-13T14:15:53.249303+0000","last_fullsized":"2024-02-13T14:15:53.249303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:12.252871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:12.252871+0000","last_clean_scrub_stamp":"2024-02-13T14:15:12.252871+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:28:21.863788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152204+0000","last_change":"2024-02-13T14:15:27.639785+0000","last_active":"2024-02-13T14:15:51.152204+0000","last_peered":"2024-02-13T14:15:51.152204+0000","last_clean":"2024-02-13T14:15:51.152204+0000","last_became_active":"2024-02-13T14:15:27.639258+0000","last_became_peered":"2024-02-13T14:15:27.639258+0000","last_unstale":"2024-02-13T14:15:51.152204+0000","last_undegraded":"2024-02-13T14:15:51.152204+0000","last_fullsized":"2024-02-13T14:15:51.152204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:57:34.751415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151350+0000","last_change":"2024-02-13T14:15:29.646852+0000","last_active":"2024-02-13T14:15:51.151350+0000","last_peered":"2024-02-13T14:15:51.151350+0000","last_clean":"2024-02-13T14:15:51.151350+0000","last_became_active":"2024-02-13T14:15:29.645962+0000","last_became_peered":"2024-02-13T14:15:29.645962+0000","last_unstale":"2024-02-13T14:15:51.151350+0000","last_undegraded":"2024-02-13T14:15:51.151350+0000","last_fullsized":"2024-02-13T14:15:51.151350+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:04:26.660676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146944+0000","last_change":"2024-02-13T14:15:25.638447+0000","last_active":"2024-02-13T14:15:51.146944+0000","last_peered":"2024-02-13T14:15:51.146944+0000","last_clean":"2024-02-13T14:15:51.146944+0000","last_became_active":"2024-02-13T14:15:25.638083+0000","last_became_peered":"2024-02-13T14:15:25.638083+0000","last_unstale":"2024-02-13T14:15:51.146944+0000","last_undegraded":"2024-02-13T14:15:51.146944+0000","last_fullsized":"2024-02-13T14:15:51.146944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:05:02.719912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149788+0000","last_change":"2024-02-13T14:15:23.621788+0000","last_active":"2024-02-13T14:15:51.149788+0000","last_peered":"2024-02-13T14:15:51.149788+0000","last_clean":"2024-02-13T14:15:51.149788+0000","last_became_active":"2024-02-13T14:15:23.621424+0000","last_became_peered":"2024-02-13T14:15:23.621424+0000","last_unstale":"2024-02-13T14:15:51.149788+0000","last_undegraded":"2024-02-13T14:15:51.149788+0000","last_fullsized":"2024-02-13T14:15:51.149788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:42:51.394977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149770+0000","last_change":"2024-02-13T14:15:27.643151+0000","last_active":"2024-02-13T14:15:51.149770+0000","last_peered":"2024-02-13T14:15:51.149770+0000","last_clean":"2024-02-13T14:15:51.149770+0000","last_became_active":"2024-02-13T14:15:27.642859+0000","last_became_peered":"2024-02-13T14:15:27.642859+0000","last_unstale":"2024-02-13T14:15:51.149770+0000","last_undegraded":"2024-02-13T14:15:51.149770+0000","last_fullsized":"2024-02-13T14:15:51.149770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:09:50.204819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151854+0000","last_change":"2024-02-13T14:15:29.646953+0000","last_active":"2024-02-13T14:15:51.151854+0000","last_peered":"2024-02-13T14:15:51.151854+0000","last_clean":"2024-02-13T14:15:51.151854+0000","last_became_active":"2024-02-13T14:15:29.646488+0000","last_became_peered":"2024-02-13T14:15:29.646488+0000","last_unstale":"2024-02-13T14:15:51.151854+0000","last_undegraded":"2024-02-13T14:15:51.151854+0000","last_fullsized":"2024-02-13T14:15:51.151854+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:44:13.181991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150132+0000","last_change":"2024-02-13T14:15:23.618522+0000","last_active":"2024-02-13T14:15:51.150132+0000","last_peered":"2024-02-13T14:15:51.150132+0000","last_clean":"2024-02-13T14:15:51.150132+0000","last_became_active":"2024-02-13T14:15:23.617267+0000","last_became_peered":"2024-02-13T14:15:23.617267+0000","last_unstale":"2024-02-13T14:15:51.150132+0000","last_undegraded":"2024-02-13T14:15:51.150132+0000","last_fullsized":"2024-02-13T14:15:51.150132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:14:37.593450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668266+0000","last_change":"2024-02-13T14:15:25.641222+0000","last_active":"2024-02-13T14:15:58.668266+0000","last_peered":"2024-02-13T14:15:58.668266+0000","last_clean":"2024-02-13T14:15:58.668266+0000","last_became_active":"2024-02-13T14:15:25.640878+0000","last_became_peered":"2024-02-13T14:15:25.640878+0000","last_unstale":"2024-02-13T14:15:58.668266+0000","last_undegraded":"2024-02-13T14:15:58.668266+0000","last_fullsized":"2024-02-13T14:15:58.668266+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:45:47.004260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668692+0000","last_change":"2024-02-13T14:15:27.641270+0000","last_active":"2024-02-13T14:15:58.668692+0000","last_peered":"2024-02-13T14:15:58.668692+0000","last_clean":"2024-02-13T14:15:58.668692+0000","last_became_active":"2024-02-13T14:15:27.639999+0000","last_became_peered":"2024-02-13T14:15:27.639999+0000","last_unstale":"2024-02-13T14:15:58.668692+0000","last_undegraded":"2024-02-13T14:15:58.668692+0000","last_fullsized":"2024-02-13T14:15:58.668692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:56:08.559879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149782+0000","last_change":"2024-02-13T14:15:29.644978+0000","last_active":"2024-02-13T14:15:51.149782+0000","last_peered":"2024-02-13T14:15:51.149782+0000","last_clean":"2024-02-13T14:15:51.149782+0000","last_became_active":"2024-02-13T14:15:29.644628+0000","last_became_peered":"2024-02-13T14:15:29.644628+0000","last_unstale":"2024-02-13T14:15:51.149782+0000","last_undegraded":"2024-02-13T14:15:51.149782+0000","last_fullsized":"2024-02-13T14:15:51.149782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:16:11.551738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149049+0000","last_change":"2024-02-13T14:15:23.622189+0000","last_active":"2024-02-13T14:15:51.149049+0000","last_peered":"2024-02-13T14:15:51.149049+0000","last_clean":"2024-02-13T14:15:51.149049+0000","last_became_active":"2024-02-13T14:15:23.621872+0000","last_became_peered":"2024-02-13T14:15:23.621872+0000","last_unstale":"2024-02-13T14:15:51.149049+0000","last_undegraded":"2024-02-13T14:15:51.149049+0000","last_fullsized":"2024-02-13T14:15:51.149049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:30:25.065018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149858+0000","last_change":"2024-02-13T14:15:25.638472+0000","last_active":"2024-02-13T14:15:51.149858+0000","last_peered":"2024-02-13T14:15:51.149858+0000","last_clean":"2024-02-13T14:15:51.149858+0000","last_became_active":"2024-02-13T14:15:25.638020+0000","last_became_peered":"2024-02-13T14:15:25.638020+0000","last_unstale":"2024-02-13T14:15:51.149858+0000","last_undegraded":"2024-02-13T14:15:51.149858+0000","last_fullsized":"2024-02-13T14:15:51.149858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:53:47.502835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668270+0000","last_change":"2024-02-13T14:15:27.642826+0000","last_active":"2024-02-13T14:15:58.668270+0000","last_peered":"2024-02-13T14:15:58.668270+0000","last_clean":"2024-02-13T14:15:58.668270+0000","last_became_active":"2024-02-13T14:15:27.642512+0000","last_became_peered":"2024-02-13T14:15:27.642512+0000","last_unstale":"2024-02-13T14:15:58.668270+0000","last_undegraded":"2024-02-13T14:15:58.668270+0000","last_fullsized":"2024-02-13T14:15:58.668270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:59:13.384374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149116+0000","last_change":"2024-02-13T14:15:29.644943+0000","last_active":"2024-02-13T14:15:51.149116+0000","last_peered":"2024-02-13T14:15:51.149116+0000","last_clean":"2024-02-13T14:15:51.149116+0000","last_became_active":"2024-02-13T14:15:29.644658+0000","last_became_peered":"2024-02-13T14:15:29.644658+0000","last_unstale":"2024-02-13T14:15:51.149116+0000","last_undegraded":"2024-02-13T14:15:51.149116+0000","last_fullsized":"2024-02-13T14:15:51.149116+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:07.119765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151834+0000","last_change":"2024-02-13T14:15:23.621272+0000","last_active":"2024-02-13T14:15:51.151834+0000","last_peered":"2024-02-13T14:15:51.151834+0000","last_clean":"2024-02-13T14:15:51.151834+0000","last_became_active":"2024-02-13T14:15:23.620828+0000","last_became_peered":"2024-02-13T14:15:23.620828+0000","last_unstale":"2024-02-13T14:15:51.151834+0000","last_undegraded":"2024-02-13T14:15:51.151834+0000","last_fullsized":"2024-02-13T14:15:51.151834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:55:36.249591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149378+0000","last_change":"2024-02-13T14:15:25.641422+0000","last_active":"2024-02-13T14:15:51.149378+0000","last_peered":"2024-02-13T14:15:51.149378+0000","last_clean":"2024-02-13T14:15:51.149378+0000","last_became_active":"2024-02-13T14:15:25.640836+0000","last_became_peered":"2024-02-13T14:15:25.640836+0000","last_unstale":"2024-02-13T14:15:51.149378+0000","last_undegraded":"2024-02-13T14:15:51.149378+0000","last_fullsized":"2024-02-13T14:15:51.149378+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:30:26.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148145+0000","last_change":"2024-02-13T14:15:27.642548+0000","last_active":"2024-02-13T14:15:51.148145+0000","last_peered":"2024-02-13T14:15:51.148145+0000","last_clean":"2024-02-13T14:15:51.148145+0000","last_became_active":"2024-02-13T14:15:27.642134+0000","last_became_peered":"2024-02-13T14:15:27.642134+0000","last_unstale":"2024-02-13T14:15:51.148145+0000","last_undegraded":"2024-02-13T14:15:51.148145+0000","last_fullsized":"2024-02-13T14:15:51.148145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:12:09.865933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151489+0000","last_change":"2024-02-13T14:15:29.647395+0000","last_active":"2024-02-13T14:15:51.151489+0000","last_peered":"2024-02-13T14:15:51.151489+0000","last_clean":"2024-02-13T14:15:51.151489+0000","last_became_active":"2024-02-13T14:15:29.646818+0000","last_became_peered":"2024-02-13T14:15:29.646818+0000","last_unstale":"2024-02-13T14:15:51.151489+0000","last_undegraded":"2024-02-13T14:15:51.151489+0000","last_fullsized":"2024-02-13T14:15:51.151489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:00:29.235602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668352+0000","last_change":"2024-02-13T14:15:27.640960+0000","last_active":"2024-02-13T14:15:58.668352+0000","last_peered":"2024-02-13T14:15:58.668352+0000","last_clean":"2024-02-13T14:15:58.668352+0000","last_became_active":"2024-02-13T14:15:27.640521+0000","last_became_peered":"2024-02-13T14:15:27.640521+0000","last_unstale":"2024-02-13T14:15:58.668352+0000","last_undegraded":"2024-02-13T14:15:58.668352+0000","last_fullsized":"2024-02-13T14:15:58.668352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:08.223607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149649+0000","last_change":"2024-02-13T14:15:23.619792+0000","last_active":"2024-02-13T14:15:51.149649+0000","last_peered":"2024-02-13T14:15:51.149649+0000","last_clean":"2024-02-13T14:15:51.149649+0000","last_became_active":"2024-02-13T14:15:23.619485+0000","last_became_peered":"2024-02-13T14:15:23.619485+0000","last_unstale":"2024-02-13T14:15:51.149649+0000","last_undegraded":"2024-02-13T14:15:51.149649+0000","last_fullsized":"2024-02-13T14:15:51.149649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:32:43.743066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149263+0000","last_change":"2024-02-13T14:15:25.631908+0000","last_active":"2024-02-13T14:15:51.149263+0000","last_peered":"2024-02-13T14:15:51.149263+0000","last_clean":"2024-02-13T14:15:51.149263+0000","last_became_active":"2024-02-13T14:15:25.631547+0000","last_became_peered":"2024-02-13T14:15:25.631547+0000","last_unstale":"2024-02-13T14:15:51.149263+0000","last_undegraded":"2024-02-13T14:15:51.149263+0000","last_fullsized":"2024-02-13T14:15:51.149263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:02:31.023771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150223+0000","last_change":"2024-02-13T14:15:29.644991+0000","last_active":"2024-02-13T14:15:51.150223+0000","last_peered":"2024-02-13T14:15:51.150223+0000","last_clean":"2024-02-13T14:15:51.150223+0000","last_became_active":"2024-02-13T14:15:29.644689+0000","last_became_peered":"2024-02-13T14:15:29.644689+0000","last_unstale":"2024-02-13T14:15:51.150223+0000","last_undegraded":"2024-02-13T14:15:51.150223+0000","last_fullsized":"2024-02-13T14:15:51.150223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:03.551408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149988+0000","last_change":"2024-02-13T14:15:27.645507+0000","last_active":"2024-02-13T14:15:51.149988+0000","last_peered":"2024-02-13T14:15:51.149988+0000","last_clean":"2024-02-13T14:15:51.149988+0000","last_became_active":"2024-02-13T14:15:27.644436+0000","last_became_peered":"2024-02-13T14:15:27.644436+0000","last_unstale":"2024-02-13T14:15:51.149988+0000","last_undegraded":"2024-02-13T14:15:51.149988+0000","last_fullsized":"2024-02-13T14:15:51.149988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:24:30.862107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146762+0000","last_change":"2024-02-13T14:15:23.621592+0000","last_active":"2024-02-13T14:15:51.146762+0000","last_peered":"2024-02-13T14:15:51.146762+0000","last_clean":"2024-02-13T14:15:51.146762+0000","last_became_active":"2024-02-13T14:15:23.621200+0000","last_became_peered":"2024-02-13T14:15:23.621200+0000","last_unstale":"2024-02-13T14:15:51.146762+0000","last_undegraded":"2024-02-13T14:15:51.146762+0000","last_fullsized":"2024-02-13T14:15:51.146762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:25:21.143290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148705+0000","last_change":"2024-02-13T14:15:25.632253+0000","last_active":"2024-02-13T14:15:51.148705+0000","last_peered":"2024-02-13T14:15:51.148705+0000","last_clean":"2024-02-13T14:15:51.148705+0000","last_became_active":"2024-02-13T14:15:25.631995+0000","last_became_peered":"2024-02-13T14:15:25.631995+0000","last_unstale":"2024-02-13T14:15:51.148705+0000","last_undegraded":"2024-02-13T14:15:51.148705+0000","last_fullsized":"2024-02-13T14:15:51.148705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:34:53.032935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149678+0000","last_change":"2024-02-13T14:15:29.643765+0000","last_active":"2024-02-13T14:15:51.149678+0000","last_peered":"2024-02-13T14:15:51.149678+0000","last_clean":"2024-02-13T14:15:51.149678+0000","last_became_active":"2024-02-13T14:15:29.643480+0000","last_became_peered":"2024-02-13T14:15:29.643480+0000","last_unstale":"2024-02-13T14:15:51.149678+0000","last_undegraded":"2024-02-13T14:15:51.149678+0000","last_fullsized":"2024-02-13T14:15:51.149678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:28:04.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148896+0000","last_change":"2024-02-13T14:15:23.616944+0000","last_active":"2024-02-13T14:15:51.148896+0000","last_peered":"2024-02-13T14:15:51.148896+0000","last_clean":"2024-02-13T14:15:51.148896+0000","last_became_active":"2024-02-13T14:15:23.616074+0000","last_became_peered":"2024-02-13T14:15:23.616074+0000","last_unstale":"2024-02-13T14:15:51.148896+0000","last_undegraded":"2024-02-13T14:15:51.148896+0000","last_fullsized":"2024-02-13T14:15:51.148896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:39:33.605890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147042+0000","last_change":"2024-02-13T14:15:25.638441+0000","last_active":"2024-02-13T14:15:51.147042+0000","last_peered":"2024-02-13T14:15:51.147042+0000","last_clean":"2024-02-13T14:15:51.147042+0000","last_became_active":"2024-02-13T14:15:25.637846+0000","last_became_peered":"2024-02-13T14:15:25.637846+0000","last_unstale":"2024-02-13T14:15:51.147042+0000","last_undegraded":"2024-02-13T14:15:51.147042+0000","last_fullsized":"2024-02-13T14:15:51.147042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:02:49.262286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150985+0000","last_change":"2024-02-13T14:15:27.640026+0000","last_active":"2024-02-13T14:15:51.150985+0000","last_peered":"2024-02-13T14:15:51.150985+0000","last_clean":"2024-02-13T14:15:51.150985+0000","last_became_active":"2024-02-13T14:15:27.639157+0000","last_became_peered":"2024-02-13T14:15:27.639157+0000","last_unstale":"2024-02-13T14:15:51.150985+0000","last_undegraded":"2024-02-13T14:15:51.150985+0000","last_fullsized":"2024-02-13T14:15:51.150985+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:35:42.139535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147114+0000","last_change":"2024-02-13T14:15:29.648975+0000","last_active":"2024-02-13T14:15:51.147114+0000","last_peered":"2024-02-13T14:15:51.147114+0000","last_clean":"2024-02-13T14:15:51.147114+0000","last_became_active":"2024-02-13T14:15:29.648559+0000","last_became_peered":"2024-02-13T14:15:29.648559+0000","last_unstale":"2024-02-13T14:15:51.147114+0000","last_undegraded":"2024-02-13T14:15:51.147114+0000","last_fullsized":"2024-02-13T14:15:51.147114+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:22:12.380243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:58.668359+0000","last_change":"2024-02-13T14:15:27.640994+0000","last_active":"2024-02-13T14:15:58.668359+0000","last_peered":"2024-02-13T14:15:58.668359+0000","last_clean":"2024-02-13T14:15:58.668359+0000","last_became_active":"2024-02-13T14:15:27.640585+0000","last_became_peered":"2024-02-13T14:15:27.640585+0000","last_unstale":"2024-02-13T14:15:58.668359+0000","last_undegraded":"2024-02-13T14:15:58.668359+0000","last_fullsized":"2024-02-13T14:15:58.668359+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:28:07.893873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149119+0000","last_change":"2024-02-13T14:15:23.620047+0000","last_active":"2024-02-13T14:15:51.149119+0000","last_peered":"2024-02-13T14:15:51.149119+0000","last_clean":"2024-02-13T14:15:51.149119+0000","last_became_active":"2024-02-13T14:15:23.618628+0000","last_became_peered":"2024-02-13T14:15:23.618628+0000","last_unstale":"2024-02-13T14:15:51.149119+0000","last_undegraded":"2024-02-13T14:15:51.149119+0000","last_fullsized":"2024-02-13T14:15:51.149119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:23:16.406845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152270+0000","last_change":"2024-02-13T14:15:25.637588+0000","last_active":"2024-02-13T14:15:51.152270+0000","last_peered":"2024-02-13T14:15:51.152270+0000","last_clean":"2024-02-13T14:15:51.152270+0000","last_became_active":"2024-02-13T14:15:25.637034+0000","last_became_peered":"2024-02-13T14:15:25.637034+0000","last_unstale":"2024-02-13T14:15:51.152270+0000","last_undegraded":"2024-02-13T14:15:51.152270+0000","last_fullsized":"2024-02-13T14:15:51.152270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:20:24.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148515+0000","last_change":"2024-02-13T14:15:29.643954+0000","last_active":"2024-02-13T14:15:51.148515+0000","last_peered":"2024-02-13T14:15:51.148515+0000","last_clean":"2024-02-13T14:15:51.148515+0000","last_became_active":"2024-02-13T14:15:29.643664+0000","last_became_peered":"2024-02-13T14:15:29.643664+0000","last_unstale":"2024-02-13T14:15:51.148515+0000","last_undegraded":"2024-02-13T14:15:51.148515+0000","last_fullsized":"2024-02-13T14:15:51.148515+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:20:23.578065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150744+0000","last_change":"2024-02-13T14:15:23.618511+0000","last_active":"2024-02-13T14:15:51.150744+0000","last_peered":"2024-02-13T14:15:51.150744+0000","last_clean":"2024-02-13T14:15:51.150744+0000","last_became_active":"2024-02-13T14:15:23.617385+0000","last_became_peered":"2024-02-13T14:15:23.617385+0000","last_unstale":"2024-02-13T14:15:51.150744+0000","last_undegraded":"2024-02-13T14:15:51.150744+0000","last_fullsized":"2024-02-13T14:15:51.150744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:34:51.554706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152432+0000","last_change":"2024-02-13T14:15:25.633129+0000","last_active":"2024-02-13T14:15:51.152432+0000","last_peered":"2024-02-13T14:15:51.152432+0000","last_clean":"2024-02-13T14:15:51.152432+0000","last_became_active":"2024-02-13T14:15:25.632752+0000","last_became_peered":"2024-02-13T14:15:25.632752+0000","last_unstale":"2024-02-13T14:15:51.152432+0000","last_undegraded":"2024-02-13T14:15:51.152432+0000","last_fullsized":"2024-02-13T14:15:51.152432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:17:38.138091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151691+0000","last_change":"2024-02-13T14:15:27.641118+0000","last_active":"2024-02-13T14:15:51.151691+0000","last_peered":"2024-02-13T14:15:51.151691+0000","last_clean":"2024-02-13T14:15:51.151691+0000","last_became_active":"2024-02-13T14:15:27.640730+0000","last_became_peered":"2024-02-13T14:15:27.640730+0000","last_unstale":"2024-02-13T14:15:51.151691+0000","last_undegraded":"2024-02-13T14:15:51.151691+0000","last_fullsized":"2024-02-13T14:15:51.151691+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:16:17.826352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.145942+0000","last_change":"2024-02-13T14:15:29.648133+0000","last_active":"2024-02-13T14:15:51.145942+0000","last_peered":"2024-02-13T14:15:51.145942+0000","last_clean":"2024-02-13T14:15:51.145942+0000","last_became_active":"2024-02-13T14:15:29.647724+0000","last_became_peered":"2024-02-13T14:15:29.647724+0000","last_unstale":"2024-02-13T14:15:51.145942+0000","last_undegraded":"2024-02-13T14:15:51.145942+0000","last_fullsized":"2024-02-13T14:15:51.145942+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:23:38.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149936+0000","last_change":"2024-02-13T14:15:23.620598+0000","last_active":"2024-02-13T14:15:51.149936+0000","last_peered":"2024-02-13T14:15:51.149936+0000","last_clean":"2024-02-13T14:15:51.149936+0000","last_became_active":"2024-02-13T14:15:23.619680+0000","last_became_peered":"2024-02-13T14:15:23.619680+0000","last_unstale":"2024-02-13T14:15:51.149936+0000","last_undegraded":"2024-02-13T14:15:51.149936+0000","last_fullsized":"2024-02-13T14:15:51.149936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:35:39.930620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151309+0000","last_change":"2024-02-13T14:15:25.636342+0000","last_active":"2024-02-13T14:15:51.151309+0000","last_peered":"2024-02-13T14:15:51.151309+0000","last_clean":"2024-02-13T14:15:51.151309+0000","last_became_active":"2024-02-13T14:15:25.635863+0000","last_became_peered":"2024-02-13T14:15:25.635863+0000","last_unstale":"2024-02-13T14:15:51.151309+0000","last_undegraded":"2024-02-13T14:15:51.151309+0000","last_fullsized":"2024-02-13T14:15:51.151309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:52:35.698341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146723+0000","last_change":"2024-02-13T14:15:27.646394+0000","last_active":"2024-02-13T14:15:51.146723+0000","last_peered":"2024-02-13T14:15:51.146723+0000","last_clean":"2024-02-13T14:15:51.146723+0000","last_became_active":"2024-02-13T14:15:27.646020+0000","last_became_peered":"2024-02-13T14:15:27.646020+0000","last_unstale":"2024-02-13T14:15:51.146723+0000","last_undegraded":"2024-02-13T14:15:51.146723+0000","last_fullsized":"2024-02-13T14:15:51.146723+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:05:00.316500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149611+0000","last_change":"2024-02-13T14:15:29.646741+0000","last_active":"2024-02-13T14:15:51.149611+0000","last_peered":"2024-02-13T14:15:51.149611+0000","last_clean":"2024-02-13T14:15:51.149611+0000","last_became_active":"2024-02-13T14:15:29.646281+0000","last_became_peered":"2024-02-13T14:15:29.646281+0000","last_unstale":"2024-02-13T14:15:51.149611+0000","last_undegraded":"2024-02-13T14:15:51.149611+0000","last_fullsized":"2024-02-13T14:15:51.149611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:50:10.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149279+0000","last_change":"2024-02-13T14:15:23.623850+0000","last_active":"2024-02-13T14:15:51.149279+0000","last_peered":"2024-02-13T14:15:51.149279+0000","last_clean":"2024-02-13T14:15:51.149279+0000","last_became_active":"2024-02-13T14:15:23.623544+0000","last_became_peered":"2024-02-13T14:15:23.623544+0000","last_unstale":"2024-02-13T14:15:51.149279+0000","last_undegraded":"2024-02-13T14:15:51.149279+0000","last_fullsized":"2024-02-13T14:15:51.149279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:38:14.058405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151935+0000","last_change":"2024-02-13T14:15:25.636494+0000","last_active":"2024-02-13T14:15:51.151935+0000","last_peered":"2024-02-13T14:15:51.151935+0000","last_clean":"2024-02-13T14:15:51.151935+0000","last_became_active":"2024-02-13T14:15:25.636113+0000","last_became_peered":"2024-02-13T14:15:25.636113+0000","last_unstale":"2024-02-13T14:15:51.151935+0000","last_undegraded":"2024-02-13T14:15:51.151935+0000","last_fullsized":"2024-02-13T14:15:51.151935+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:56:12.278547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148886+0000","last_change":"2024-02-13T14:15:27.639612+0000","last_active":"2024-02-13T14:15:51.148886+0000","last_peered":"2024-02-13T14:15:51.148886+0000","last_clean":"2024-02-13T14:15:51.148886+0000","last_became_active":"2024-02-13T14:15:27.639326+0000","last_became_peered":"2024-02-13T14:15:27.639326+0000","last_unstale":"2024-02-13T14:15:51.148886+0000","last_undegraded":"2024-02-13T14:15:51.148886+0000","last_fullsized":"2024-02-13T14:15:51.148886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:51:52.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.316},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.28499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T14:16:08.731 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph pg dump --format=json 2024-02-13T14:16:09.575 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:10.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:10 smithi055 ceph-mon[25769]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:10.645 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:10 smithi078 ceph-mon[30086]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:10.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:10 smithi170 ceph-mon[30181]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:12.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:12 smithi055 ceph-mon[25769]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:12.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:12 smithi170 ceph-mon[30181]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:12.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:12 smithi078 ceph-mon[30086]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:13.280 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:13 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:16:13] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:13.631 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:16:13.634 INFO:teuthology.orchestra.run.smithi055.stderr:dumped all 2024-02-13T14:16:13.904 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:13 smithi055 podman[57432]: 2024-02-13 14:16:13.714683534 +0000 UTC m=+0.229303520 container create cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 podman[57432]: 2024-02-13 14:16:14.02537314 +0000 UTC m=+0.539993090 container init cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 podman[57432]: 2024-02-13 14:16:14.030785366 +0000 UTC m=+0.545405314 container start cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.043Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.043Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.044Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.55 port=9094 2024-02-13T14:16:14.172 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.045Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.068Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.068Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.071Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:14.071Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 bash[57432]: cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff 2024-02-13T14:16:14.173 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:14 smithi055 systemd[1]: Started Ceph alertmanager.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:14.455 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.456 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE Bus STOPPING 2024-02-13T14:16:14.456 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:16:14.456 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE Bus STOPPED 2024-02-13T14:16:14.456 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE Bus STARTING 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:14.657 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:14 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:14.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:14 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:14.721 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE Serving on http://:::9283 2024-02-13T14:16:14.721 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:14] ENGINE Bus STARTED 2024-02-13T14:16:14.747 INFO:teuthology.orchestra.run.smithi055.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-02-13T14:16:13.161314+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96616,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126552,"statfs":{"total":287972524032,"available":287873589248,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":95743311},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002851"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150719+0000","last_change":"2024-02-13T14:15:29.645879+0000","last_active":"2024-02-13T14:15:51.150719+0000","last_peered":"2024-02-13T14:15:51.150719+0000","last_clean":"2024-02-13T14:15:51.150719+0000","last_became_active":"2024-02-13T14:15:29.645337+0000","last_became_peered":"2024-02-13T14:15:29.645337+0000","last_unstale":"2024-02-13T14:15:51.150719+0000","last_undegraded":"2024-02-13T14:15:51.150719+0000","last_fullsized":"2024-02-13T14:15:51.150719+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:31:26.104665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147897+0000","last_change":"2024-02-13T14:15:27.641831+0000","last_active":"2024-02-13T14:15:51.147897+0000","last_peered":"2024-02-13T14:15:51.147897+0000","last_clean":"2024-02-13T14:15:51.147897+0000","last_became_active":"2024-02-13T14:15:27.640612+0000","last_became_peered":"2024-02-13T14:15:27.640612+0000","last_unstale":"2024-02-13T14:15:51.147897+0000","last_undegraded":"2024-02-13T14:15:51.147897+0000","last_fullsized":"2024-02-13T14:15:51.147897+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:20:21.915769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149702+0000","last_change":"2024-02-13T14:15:23.622834+0000","last_active":"2024-02-13T14:15:51.149702+0000","last_peered":"2024-02-13T14:15:51.149702+0000","last_clean":"2024-02-13T14:15:51.149702+0000","last_became_active":"2024-02-13T14:15:23.622511+0000","last_became_peered":"2024-02-13T14:15:23.622511+0000","last_unstale":"2024-02-13T14:15:51.149702+0000","last_undegraded":"2024-02-13T14:15:51.149702+0000","last_fullsized":"2024-02-13T14:15:51.149702+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:45:17.706098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149161+0000","last_change":"2024-02-13T14:15:25.640119+0000","last_active":"2024-02-13T14:15:51.149161+0000","last_peered":"2024-02-13T14:15:51.149161+0000","last_clean":"2024-02-13T14:15:51.149161+0000","last_became_active":"2024-02-13T14:15:25.639549+0000","last_became_peered":"2024-02-13T14:15:25.639549+0000","last_unstale":"2024-02-13T14:15:51.149161+0000","last_undegraded":"2024-02-13T14:15:51.149161+0000","last_fullsized":"2024-02-13T14:15:51.149161+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:11:52.229637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146710+0000","last_change":"2024-02-13T14:15:29.650084+0000","last_active":"2024-02-13T14:15:51.146710+0000","last_peered":"2024-02-13T14:15:51.146710+0000","last_clean":"2024-02-13T14:15:51.146710+0000","last_became_active":"2024-02-13T14:15:29.648790+0000","last_became_peered":"2024-02-13T14:15:29.648790+0000","last_unstale":"2024-02-13T14:15:51.146710+0000","last_undegraded":"2024-02-13T14:15:51.146710+0000","last_fullsized":"2024-02-13T14:15:51.146710+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:51:33.104466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150077+0000","last_change":"2024-02-13T14:15:27.640984+0000","last_active":"2024-02-13T14:15:51.150077+0000","last_peered":"2024-02-13T14:15:51.150077+0000","last_clean":"2024-02-13T14:15:51.150077+0000","last_became_active":"2024-02-13T14:15:27.640600+0000","last_became_peered":"2024-02-13T14:15:27.640600+0000","last_unstale":"2024-02-13T14:15:51.150077+0000","last_undegraded":"2024-02-13T14:15:51.150077+0000","last_fullsized":"2024-02-13T14:15:51.150077+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:25:20.034995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149517+0000","last_change":"2024-02-13T14:15:23.623943+0000","last_active":"2024-02-13T14:15:51.149517+0000","last_peered":"2024-02-13T14:15:51.149517+0000","last_clean":"2024-02-13T14:15:51.149517+0000","last_became_active":"2024-02-13T14:15:23.623658+0000","last_became_peered":"2024-02-13T14:15:23.623658+0000","last_unstale":"2024-02-13T14:15:51.149517+0000","last_undegraded":"2024-02-13T14:15:51.149517+0000","last_fullsized":"2024-02-13T14:15:51.149517+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:21:10.749864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149122+0000","last_change":"2024-02-13T14:15:25.639691+0000","last_active":"2024-02-13T14:15:51.149122+0000","last_peered":"2024-02-13T14:15:51.149122+0000","last_clean":"2024-02-13T14:15:51.149122+0000","last_became_active":"2024-02-13T14:15:25.639293+0000","last_became_peered":"2024-02-13T14:15:25.639293+0000","last_unstale":"2024-02-13T14:15:51.149122+0000","last_undegraded":"2024-02-13T14:15:51.149122+0000","last_fullsized":"2024-02-13T14:15:51.149122+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:39:09.443491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148115+0000","last_change":"2024-02-13T14:15:29.645303+0000","last_active":"2024-02-13T14:15:51.148115+0000","last_peered":"2024-02-13T14:15:51.148115+0000","last_clean":"2024-02-13T14:15:51.148115+0000","last_became_active":"2024-02-13T14:15:29.644992+0000","last_became_peered":"2024-02-13T14:15:29.644992+0000","last_unstale":"2024-02-13T14:15:51.148115+0000","last_undegraded":"2024-02-13T14:15:51.148115+0000","last_fullsized":"2024-02-13T14:15:51.148115+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:55:36.492078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150112+0000","last_change":"2024-02-13T14:15:27.640101+0000","last_active":"2024-02-13T14:15:51.150112+0000","last_peered":"2024-02-13T14:15:51.150112+0000","last_clean":"2024-02-13T14:15:51.150112+0000","last_became_active":"2024-02-13T14:15:27.639712+0000","last_became_peered":"2024-02-13T14:15:27.639712+0000","last_unstale":"2024-02-13T14:15:51.150112+0000","last_undegraded":"2024-02-13T14:15:51.150112+0000","last_fullsized":"2024-02-13T14:15:51.150112+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:49.267601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150547+0000","last_change":"2024-02-13T14:15:23.618714+0000","last_active":"2024-02-13T14:15:51.150547+0000","last_peered":"2024-02-13T14:15:51.150547+0000","last_clean":"2024-02-13T14:15:51.150547+0000","last_became_active":"2024-02-13T14:15:23.617832+0000","last_became_peered":"2024-02-13T14:15:23.617832+0000","last_unstale":"2024-02-13T14:15:51.150547+0000","last_undegraded":"2024-02-13T14:15:51.150547+0000","last_fullsized":"2024-02-13T14:15:51.150547+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:35:15.756097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151435+0000","last_change":"2024-02-13T14:15:25.632979+0000","last_active":"2024-02-13T14:15:51.151435+0000","last_peered":"2024-02-13T14:15:51.151435+0000","last_clean":"2024-02-13T14:15:51.151435+0000","last_became_active":"2024-02-13T14:15:25.632544+0000","last_became_peered":"2024-02-13T14:15:25.632544+0000","last_unstale":"2024-02-13T14:15:51.151435+0000","last_undegraded":"2024-02-13T14:15:51.151435+0000","last_fullsized":"2024-02-13T14:15:51.151435+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:49:06.988964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149918+0000","last_change":"2024-02-13T14:15:29.646777+0000","last_active":"2024-02-13T14:15:51.149918+0000","last_peered":"2024-02-13T14:15:51.149918+0000","last_clean":"2024-02-13T14:15:51.149918+0000","last_became_active":"2024-02-13T14:15:29.646028+0000","last_became_peered":"2024-02-13T14:15:29.646028+0000","last_unstale":"2024-02-13T14:15:51.149918+0000","last_undegraded":"2024-02-13T14:15:51.149918+0000","last_fullsized":"2024-02-13T14:15:51.149918+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:03:35.458796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150033+0000","last_change":"2024-02-13T14:15:27.640177+0000","last_active":"2024-02-13T14:15:51.150033+0000","last_peered":"2024-02-13T14:15:51.150033+0000","last_clean":"2024-02-13T14:15:51.150033+0000","last_became_active":"2024-02-13T14:15:27.639796+0000","last_became_peered":"2024-02-13T14:15:27.639796+0000","last_unstale":"2024-02-13T14:15:51.150033+0000","last_undegraded":"2024-02-13T14:15:51.150033+0000","last_fullsized":"2024-02-13T14:15:51.150033+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:19:19.440771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150558+0000","last_change":"2024-02-13T14:15:23.617880+0000","last_active":"2024-02-13T14:15:51.150558+0000","last_peered":"2024-02-13T14:15:51.150558+0000","last_clean":"2024-02-13T14:15:51.150558+0000","last_became_active":"2024-02-13T14:15:23.617098+0000","last_became_peered":"2024-02-13T14:15:23.617098+0000","last_unstale":"2024-02-13T14:15:51.150558+0000","last_undegraded":"2024-02-13T14:15:51.150558+0000","last_fullsized":"2024-02-13T14:15:51.150558+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:47:43.267755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150338+0000","last_change":"2024-02-13T14:15:25.632775+0000","last_active":"2024-02-13T14:15:51.150338+0000","last_peered":"2024-02-13T14:15:51.150338+0000","last_clean":"2024-02-13T14:15:51.150338+0000","last_became_active":"2024-02-13T14:15:25.632470+0000","last_became_peered":"2024-02-13T14:15:25.632470+0000","last_unstale":"2024-02-13T14:15:51.150338+0000","last_undegraded":"2024-02-13T14:15:51.150338+0000","last_fullsized":"2024-02-13T14:15:51.150338+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:19:18.279751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150858+0000","last_change":"2024-02-13T14:15:29.647903+0000","last_active":"2024-02-13T14:15:51.150858+0000","last_peered":"2024-02-13T14:15:51.150858+0000","last_clean":"2024-02-13T14:15:51.150858+0000","last_became_active":"2024-02-13T14:15:29.647003+0000","last_became_peered":"2024-02-13T14:15:29.647003+0000","last_unstale":"2024-02-13T14:15:51.150858+0000","last_undegraded":"2024-02-13T14:15:51.150858+0000","last_fullsized":"2024-02-13T14:15:51.150858+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:05:46.480579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149686+0000","last_change":"2024-02-13T14:15:27.641240+0000","last_active":"2024-02-13T14:15:51.149686+0000","last_peered":"2024-02-13T14:15:51.149686+0000","last_clean":"2024-02-13T14:15:51.149686+0000","last_became_active":"2024-02-13T14:15:27.640912+0000","last_became_peered":"2024-02-13T14:15:27.640912+0000","last_unstale":"2024-02-13T14:15:51.149686+0000","last_undegraded":"2024-02-13T14:15:51.149686+0000","last_fullsized":"2024-02-13T14:15:51.149686+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:36:27.153329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149324+0000","last_change":"2024-02-13T14:15:23.619731+0000","last_active":"2024-02-13T14:15:51.149324+0000","last_peered":"2024-02-13T14:15:51.149324+0000","last_clean":"2024-02-13T14:15:51.149324+0000","last_became_active":"2024-02-13T14:15:23.618669+0000","last_became_peered":"2024-02-13T14:15:23.618669+0000","last_unstale":"2024-02-13T14:15:51.149324+0000","last_undegraded":"2024-02-13T14:15:51.149324+0000","last_fullsized":"2024-02-13T14:15:51.149324+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:01:40.159041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148929+0000","last_change":"2024-02-13T14:15:25.632087+0000","last_active":"2024-02-13T14:15:51.148929+0000","last_peered":"2024-02-13T14:15:51.148929+0000","last_clean":"2024-02-13T14:15:51.148929+0000","last_became_active":"2024-02-13T14:15:25.631709+0000","last_became_peered":"2024-02-13T14:15:25.631709+0000","last_unstale":"2024-02-13T14:15:51.148929+0000","last_undegraded":"2024-02-13T14:15:51.148929+0000","last_fullsized":"2024-02-13T14:15:51.148929+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:54:53.929894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148469+0000","last_change":"2024-02-13T14:15:29.650817+0000","last_active":"2024-02-13T14:15:51.148469+0000","last_peered":"2024-02-13T14:15:51.148469+0000","last_clean":"2024-02-13T14:15:51.148469+0000","last_became_active":"2024-02-13T14:15:29.650352+0000","last_became_peered":"2024-02-13T14:15:29.650352+0000","last_unstale":"2024-02-13T14:15:51.148469+0000","last_undegraded":"2024-02-13T14:15:51.148469+0000","last_fullsized":"2024-02-13T14:15:51.148469+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:26:07.103058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150352+0000","last_change":"2024-02-13T14:15:27.641910+0000","last_active":"2024-02-13T14:15:51.150352+0000","last_peered":"2024-02-13T14:15:51.150352+0000","last_clean":"2024-02-13T14:15:51.150352+0000","last_became_active":"2024-02-13T14:15:27.641619+0000","last_became_peered":"2024-02-13T14:15:27.641619+0000","last_unstale":"2024-02-13T14:15:51.150352+0000","last_undegraded":"2024-02-13T14:15:51.150352+0000","last_fullsized":"2024-02-13T14:15:51.150352+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:50:21.787723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151955+0000","last_change":"2024-02-13T14:15:23.619919+0000","last_active":"2024-02-13T14:15:51.151955+0000","last_peered":"2024-02-13T14:15:51.151955+0000","last_clean":"2024-02-13T14:15:51.151955+0000","last_became_active":"2024-02-13T14:15:23.619361+0000","last_became_peered":"2024-02-13T14:15:23.619361+0000","last_unstale":"2024-02-13T14:15:51.151955+0000","last_undegraded":"2024-02-13T14:15:51.151955+0000","last_fullsized":"2024-02-13T14:15:51.151955+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:47:48.518820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149003+0000","last_change":"2024-02-13T14:15:25.631457+0000","last_active":"2024-02-13T14:15:51.149003+0000","last_peered":"2024-02-13T14:15:51.149003+0000","last_clean":"2024-02-13T14:15:51.149003+0000","last_became_active":"2024-02-13T14:15:25.631063+0000","last_became_peered":"2024-02-13T14:15:25.631063+0000","last_unstale":"2024-02-13T14:15:51.149003+0000","last_undegraded":"2024-02-13T14:15:51.149003+0000","last_fullsized":"2024-02-13T14:15:51.149003+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:45:55.797820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149338+0000","last_change":"2024-02-13T14:15:29.648463+0000","last_active":"2024-02-13T14:15:51.149338+0000","last_peered":"2024-02-13T14:15:51.149338+0000","last_clean":"2024-02-13T14:15:51.149338+0000","last_became_active":"2024-02-13T14:15:29.647919+0000","last_became_peered":"2024-02-13T14:15:29.647919+0000","last_unstale":"2024-02-13T14:15:51.149338+0000","last_undegraded":"2024-02-13T14:15:51.149338+0000","last_fullsized":"2024-02-13T14:15:51.149338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:02:23.090634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669651+0000","last_change":"2024-02-13T14:15:27.643210+0000","last_active":"2024-02-13T14:16:03.669651+0000","last_peered":"2024-02-13T14:16:03.669651+0000","last_clean":"2024-02-13T14:16:03.669651+0000","last_became_active":"2024-02-13T14:15:27.642379+0000","last_became_peered":"2024-02-13T14:15:27.642379+0000","last_unstale":"2024-02-13T14:16:03.669651+0000","last_undegraded":"2024-02-13T14:16:03.669651+0000","last_fullsized":"2024-02-13T14:16:03.669651+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:46:12.920991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146895+0000","last_change":"2024-02-13T14:15:23.622973+0000","last_active":"2024-02-13T14:15:51.146895+0000","last_peered":"2024-02-13T14:15:51.146895+0000","last_clean":"2024-02-13T14:15:51.146895+0000","last_became_active":"2024-02-13T14:15:23.622680+0000","last_became_peered":"2024-02-13T14:15:23.622680+0000","last_unstale":"2024-02-13T14:15:51.146895+0000","last_undegraded":"2024-02-13T14:15:51.146895+0000","last_fullsized":"2024-02-13T14:15:51.146895+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:42:39.816463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146083+0000","last_change":"2024-02-13T14:15:25.638424+0000","last_active":"2024-02-13T14:15:51.146083+0000","last_peered":"2024-02-13T14:15:51.146083+0000","last_clean":"2024-02-13T14:15:51.146083+0000","last_became_active":"2024-02-13T14:15:25.638015+0000","last_became_peered":"2024-02-13T14:15:25.638015+0000","last_unstale":"2024-02-13T14:15:51.146083+0000","last_undegraded":"2024-02-13T14:15:51.146083+0000","last_fullsized":"2024-02-13T14:15:51.146083+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:35:10.188869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151238+0000","last_change":"2024-02-13T14:15:29.648091+0000","last_active":"2024-02-13T14:15:51.151238+0000","last_peered":"2024-02-13T14:15:51.151238+0000","last_clean":"2024-02-13T14:15:51.151238+0000","last_became_active":"2024-02-13T14:15:29.647683+0000","last_became_peered":"2024-02-13T14:15:29.647683+0000","last_unstale":"2024-02-13T14:15:51.151238+0000","last_undegraded":"2024-02-13T14:15:51.151238+0000","last_fullsized":"2024-02-13T14:15:51.151238+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:54:06.592494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148769+0000","last_change":"2024-02-13T14:15:27.643425+0000","last_active":"2024-02-13T14:15:51.148769+0000","last_peered":"2024-02-13T14:15:51.148769+0000","last_clean":"2024-02-13T14:15:51.148769+0000","last_became_active":"2024-02-13T14:15:27.643068+0000","last_became_peered":"2024-02-13T14:15:27.643068+0000","last_unstale":"2024-02-13T14:15:51.148769+0000","last_undegraded":"2024-02-13T14:15:51.148769+0000","last_fullsized":"2024-02-13T14:15:51.148769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:32:14.274046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148026+0000","last_change":"2024-02-13T14:15:23.620818+0000","last_active":"2024-02-13T14:15:51.148026+0000","last_peered":"2024-02-13T14:15:51.148026+0000","last_clean":"2024-02-13T14:15:51.148026+0000","last_became_active":"2024-02-13T14:15:23.620520+0000","last_became_peered":"2024-02-13T14:15:23.620520+0000","last_unstale":"2024-02-13T14:15:51.148026+0000","last_undegraded":"2024-02-13T14:15:51.148026+0000","last_fullsized":"2024-02-13T14:15:51.148026+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:18.701338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146464+0000","last_change":"2024-02-13T14:15:25.639838+0000","last_active":"2024-02-13T14:15:51.146464+0000","last_peered":"2024-02-13T14:15:51.146464+0000","last_clean":"2024-02-13T14:15:51.146464+0000","last_became_active":"2024-02-13T14:15:25.639119+0000","last_became_peered":"2024-02-13T14:15:25.639119+0000","last_unstale":"2024-02-13T14:15:51.146464+0000","last_undegraded":"2024-02-13T14:15:51.146464+0000","last_fullsized":"2024-02-13T14:15:51.146464+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:59:13.268372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151147+0000","last_change":"2024-02-13T14:15:29.648555+0000","last_active":"2024-02-13T14:15:51.151147+0000","last_peered":"2024-02-13T14:15:51.151147+0000","last_clean":"2024-02-13T14:15:51.151147+0000","last_became_active":"2024-02-13T14:15:29.647901+0000","last_became_peered":"2024-02-13T14:15:29.647901+0000","last_unstale":"2024-02-13T14:15:51.151147+0000","last_undegraded":"2024-02-13T14:15:51.151147+0000","last_fullsized":"2024-02-13T14:15:51.151147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:08:58.959202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146147+0000","last_change":"2024-02-13T14:15:27.645059+0000","last_active":"2024-02-13T14:15:51.146147+0000","last_peered":"2024-02-13T14:15:51.146147+0000","last_clean":"2024-02-13T14:15:51.146147+0000","last_became_active":"2024-02-13T14:15:27.644421+0000","last_became_peered":"2024-02-13T14:15:27.644421+0000","last_unstale":"2024-02-13T14:15:51.146147+0000","last_undegraded":"2024-02-13T14:15:51.146147+0000","last_fullsized":"2024-02-13T14:15:51.146147+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:03:59.473260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152081+0000","last_change":"2024-02-13T14:15:23.620877+0000","last_active":"2024-02-13T14:15:51.152081+0000","last_peered":"2024-02-13T14:15:51.152081+0000","last_clean":"2024-02-13T14:15:51.152081+0000","last_became_active":"2024-02-13T14:15:23.620356+0000","last_became_peered":"2024-02-13T14:15:23.620356+0000","last_unstale":"2024-02-13T14:15:51.152081+0000","last_undegraded":"2024-02-13T14:15:51.152081+0000","last_fullsized":"2024-02-13T14:15:51.152081+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:33.686661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151256+0000","last_change":"2024-02-13T14:15:25.636137+0000","last_active":"2024-02-13T14:15:51.151256+0000","last_peered":"2024-02-13T14:15:51.151256+0000","last_clean":"2024-02-13T14:15:51.151256+0000","last_became_active":"2024-02-13T14:15:25.635594+0000","last_became_peered":"2024-02-13T14:15:25.635594+0000","last_unstale":"2024-02-13T14:15:51.151256+0000","last_undegraded":"2024-02-13T14:15:51.151256+0000","last_fullsized":"2024-02-13T14:15:51.151256+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:41.070936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150310+0000","last_change":"2024-02-13T14:15:29.645215+0000","last_active":"2024-02-13T14:15:51.150310+0000","last_peered":"2024-02-13T14:15:51.150310+0000","last_clean":"2024-02-13T14:15:51.150310+0000","last_became_active":"2024-02-13T14:15:29.644869+0000","last_became_peered":"2024-02-13T14:15:29.644869+0000","last_unstale":"2024-02-13T14:15:51.150310+0000","last_undegraded":"2024-02-13T14:15:51.150310+0000","last_fullsized":"2024-02-13T14:15:51.150310+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:49:49.161277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.145938+0000","last_change":"2024-02-13T14:15:27.643536+0000","last_active":"2024-02-13T14:15:51.145938+0000","last_peered":"2024-02-13T14:15:51.145938+0000","last_clean":"2024-02-13T14:15:51.145938+0000","last_became_active":"2024-02-13T14:15:27.643173+0000","last_became_peered":"2024-02-13T14:15:27.643173+0000","last_unstale":"2024-02-13T14:15:51.145938+0000","last_undegraded":"2024-02-13T14:15:51.145938+0000","last_fullsized":"2024-02-13T14:15:51.145938+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:04:45.289058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150925+0000","last_change":"2024-02-13T14:15:23.621820+0000","last_active":"2024-02-13T14:15:51.150925+0000","last_peered":"2024-02-13T14:15:51.150925+0000","last_clean":"2024-02-13T14:15:51.150925+0000","last_became_active":"2024-02-13T14:15:23.621457+0000","last_became_peered":"2024-02-13T14:15:23.621457+0000","last_unstale":"2024-02-13T14:15:51.150925+0000","last_undegraded":"2024-02-13T14:15:51.150925+0000","last_fullsized":"2024-02-13T14:15:51.150925+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:03:21.989986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149384+0000","last_change":"2024-02-13T14:15:25.640447+0000","last_active":"2024-02-13T14:15:51.149384+0000","last_peered":"2024-02-13T14:15:51.149384+0000","last_clean":"2024-02-13T14:15:51.149384+0000","last_became_active":"2024-02-13T14:15:25.638041+0000","last_became_peered":"2024-02-13T14:15:25.638041+0000","last_unstale":"2024-02-13T14:15:51.149384+0000","last_undegraded":"2024-02-13T14:15:51.149384+0000","last_fullsized":"2024-02-13T14:15:51.149384+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:43:48.962133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148337+0000","last_change":"2024-02-13T14:15:29.644948+0000","last_active":"2024-02-13T14:15:51.148337+0000","last_peered":"2024-02-13T14:15:51.148337+0000","last_clean":"2024-02-13T14:15:51.148337+0000","last_became_active":"2024-02-13T14:15:29.644522+0000","last_became_peered":"2024-02-13T14:15:29.644522+0000","last_unstale":"2024-02-13T14:15:51.148337+0000","last_undegraded":"2024-02-13T14:15:51.148337+0000","last_fullsized":"2024-02-13T14:15:51.148337+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:58:47.616179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150285+0000","last_change":"2024-02-13T14:15:27.640695+0000","last_active":"2024-02-13T14:15:51.150285+0000","last_peered":"2024-02-13T14:15:51.150285+0000","last_clean":"2024-02-13T14:15:51.150285+0000","last_became_active":"2024-02-13T14:15:27.640342+0000","last_became_peered":"2024-02-13T14:15:27.640342+0000","last_unstale":"2024-02-13T14:15:51.150285+0000","last_undegraded":"2024-02-13T14:15:51.150285+0000","last_fullsized":"2024-02-13T14:15:51.150285+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:24:42.246645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150236+0000","last_change":"2024-02-13T14:15:23.620111+0000","last_active":"2024-02-13T14:15:51.150236+0000","last_peered":"2024-02-13T14:15:51.150236+0000","last_clean":"2024-02-13T14:15:51.150236+0000","last_became_active":"2024-02-13T14:15:23.618976+0000","last_became_peered":"2024-02-13T14:15:23.618976+0000","last_unstale":"2024-02-13T14:15:51.150236+0000","last_undegraded":"2024-02-13T14:15:51.150236+0000","last_fullsized":"2024-02-13T14:15:51.150236+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:38:48.463427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151773+0000","last_change":"2024-02-13T14:15:25.634282+0000","last_active":"2024-02-13T14:15:51.151773+0000","last_peered":"2024-02-13T14:15:51.151773+0000","last_clean":"2024-02-13T14:15:51.151773+0000","last_became_active":"2024-02-13T14:15:25.633915+0000","last_became_peered":"2024-02-13T14:15:25.633915+0000","last_unstale":"2024-02-13T14:15:51.151773+0000","last_undegraded":"2024-02-13T14:15:51.151773+0000","last_fullsized":"2024-02-13T14:15:51.151773+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:29:51.823704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148330+0000","last_change":"2024-02-13T14:15:29.644167+0000","last_active":"2024-02-13T14:15:51.148330+0000","last_peered":"2024-02-13T14:15:51.148330+0000","last_clean":"2024-02-13T14:15:51.148330+0000","last_became_active":"2024-02-13T14:15:29.643846+0000","last_became_peered":"2024-02-13T14:15:29.643846+0000","last_unstale":"2024-02-13T14:15:51.148330+0000","last_undegraded":"2024-02-13T14:15:51.148330+0000","last_fullsized":"2024-02-13T14:15:51.148330+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:14:08.875512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146242+0000","last_change":"2024-02-13T14:15:27.642965+0000","last_active":"2024-02-13T14:15:51.146242+0000","last_peered":"2024-02-13T14:15:51.146242+0000","last_clean":"2024-02-13T14:15:51.146242+0000","last_became_active":"2024-02-13T14:15:27.642585+0000","last_became_peered":"2024-02-13T14:15:27.642585+0000","last_unstale":"2024-02-13T14:15:51.146242+0000","last_undegraded":"2024-02-13T14:15:51.146242+0000","last_fullsized":"2024-02-13T14:15:51.146242+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:48:25.621534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151204+0000","last_change":"2024-02-13T14:15:23.620750+0000","last_active":"2024-02-13T14:15:51.151204+0000","last_peered":"2024-02-13T14:15:51.151204+0000","last_clean":"2024-02-13T14:15:51.151204+0000","last_became_active":"2024-02-13T14:15:23.620347+0000","last_became_peered":"2024-02-13T14:15:23.620347+0000","last_unstale":"2024-02-13T14:15:51.151204+0000","last_undegraded":"2024-02-13T14:15:51.151204+0000","last_fullsized":"2024-02-13T14:15:51.151204+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:06:33.085887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150069+0000","last_change":"2024-02-13T14:15:25.632939+0000","last_active":"2024-02-13T14:15:51.150069+0000","last_peered":"2024-02-13T14:15:51.150069+0000","last_clean":"2024-02-13T14:15:51.150069+0000","last_became_active":"2024-02-13T14:15:25.632560+0000","last_became_peered":"2024-02-13T14:15:25.632560+0000","last_unstale":"2024-02-13T14:15:51.150069+0000","last_undegraded":"2024-02-13T14:15:51.150069+0000","last_fullsized":"2024-02-13T14:15:51.150069+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:51:13.680314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147259+0000","last_change":"2024-02-13T14:15:29.649650+0000","last_active":"2024-02-13T14:15:51.147259+0000","last_peered":"2024-02-13T14:15:51.147259+0000","last_clean":"2024-02-13T14:15:51.147259+0000","last_became_active":"2024-02-13T14:15:29.649249+0000","last_became_peered":"2024-02-13T14:15:29.649249+0000","last_unstale":"2024-02-13T14:15:51.147259+0000","last_undegraded":"2024-02-13T14:15:51.147259+0000","last_fullsized":"2024-02-13T14:15:51.147259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:48:39.046537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669693+0000","last_change":"2024-02-13T14:15:27.643637+0000","last_active":"2024-02-13T14:16:03.669693+0000","last_peered":"2024-02-13T14:16:03.669693+0000","last_clean":"2024-02-13T14:16:03.669693+0000","last_became_active":"2024-02-13T14:15:27.643224+0000","last_became_peered":"2024-02-13T14:15:27.643224+0000","last_unstale":"2024-02-13T14:16:03.669693+0000","last_undegraded":"2024-02-13T14:16:03.669693+0000","last_fullsized":"2024-02-13T14:16:03.669693+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:33:48.279038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149508+0000","last_change":"2024-02-13T14:15:23.618907+0000","last_active":"2024-02-13T14:15:51.149508+0000","last_peered":"2024-02-13T14:15:51.149508+0000","last_clean":"2024-02-13T14:15:51.149508+0000","last_became_active":"2024-02-13T14:15:23.617121+0000","last_became_peered":"2024-02-13T14:15:23.617121+0000","last_unstale":"2024-02-13T14:15:51.149508+0000","last_undegraded":"2024-02-13T14:15:51.149508+0000","last_fullsized":"2024-02-13T14:15:51.149508+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:44:58.554123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146427+0000","last_change":"2024-02-13T14:15:25.642435+0000","last_active":"2024-02-13T14:15:51.146427+0000","last_peered":"2024-02-13T14:15:51.146427+0000","last_clean":"2024-02-13T14:15:51.146427+0000","last_became_active":"2024-02-13T14:15:25.642078+0000","last_became_peered":"2024-02-13T14:15:25.642078+0000","last_unstale":"2024-02-13T14:15:51.146427+0000","last_undegraded":"2024-02-13T14:15:51.146427+0000","last_fullsized":"2024-02-13T14:15:51.146427+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:45:06.913112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151566+0000","last_change":"2024-02-13T14:15:29.647394+0000","last_active":"2024-02-13T14:15:51.151566+0000","last_peered":"2024-02-13T14:15:51.151566+0000","last_clean":"2024-02-13T14:15:51.151566+0000","last_became_active":"2024-02-13T14:15:29.646158+0000","last_became_peered":"2024-02-13T14:15:29.646158+0000","last_unstale":"2024-02-13T14:15:51.151566+0000","last_undegraded":"2024-02-13T14:15:51.151566+0000","last_fullsized":"2024-02-13T14:15:51.151566+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:54:51.741665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669671+0000","last_change":"2024-02-13T14:15:27.641066+0000","last_active":"2024-02-13T14:16:03.669671+0000","last_peered":"2024-02-13T14:16:03.669671+0000","last_clean":"2024-02-13T14:16:03.669671+0000","last_became_active":"2024-02-13T14:15:27.640758+0000","last_became_peered":"2024-02-13T14:15:27.640758+0000","last_unstale":"2024-02-13T14:16:03.669671+0000","last_undegraded":"2024-02-13T14:16:03.669671+0000","last_fullsized":"2024-02-13T14:16:03.669671+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:21:49.097191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148618+0000","last_change":"2024-02-13T14:15:23.618464+0000","last_active":"2024-02-13T14:15:51.148618+0000","last_peered":"2024-02-13T14:15:51.148618+0000","last_clean":"2024-02-13T14:15:51.148618+0000","last_became_active":"2024-02-13T14:15:23.617343+0000","last_became_peered":"2024-02-13T14:15:23.617343+0000","last_unstale":"2024-02-13T14:15:51.148618+0000","last_undegraded":"2024-02-13T14:15:51.148618+0000","last_fullsized":"2024-02-13T14:15:51.148618+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:41:01.716785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150656+0000","last_change":"2024-02-13T14:15:25.633331+0000","last_active":"2024-02-13T14:15:51.150656+0000","last_peered":"2024-02-13T14:15:51.150656+0000","last_clean":"2024-02-13T14:15:51.150656+0000","last_became_active":"2024-02-13T14:15:25.632968+0000","last_became_peered":"2024-02-13T14:15:25.632968+0000","last_unstale":"2024-02-13T14:15:51.150656+0000","last_undegraded":"2024-02-13T14:15:51.150656+0000","last_fullsized":"2024-02-13T14:15:51.150656+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:41:14.441961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150373+0000","last_change":"2024-02-13T14:15:29.647789+0000","last_active":"2024-02-13T14:15:51.150373+0000","last_peered":"2024-02-13T14:15:51.150373+0000","last_clean":"2024-02-13T14:15:51.150373+0000","last_became_active":"2024-02-13T14:15:29.647388+0000","last_became_peered":"2024-02-13T14:15:29.647388+0000","last_unstale":"2024-02-13T14:15:51.150373+0000","last_undegraded":"2024-02-13T14:15:51.150373+0000","last_fullsized":"2024-02-13T14:15:51.150373+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:58:44.557944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149021+0000","last_change":"2024-02-13T14:15:27.645782+0000","last_active":"2024-02-13T14:15:51.149021+0000","last_peered":"2024-02-13T14:15:51.149021+0000","last_clean":"2024-02-13T14:15:51.149021+0000","last_became_active":"2024-02-13T14:15:27.644997+0000","last_became_peered":"2024-02-13T14:15:27.644997+0000","last_unstale":"2024-02-13T14:15:51.149021+0000","last_undegraded":"2024-02-13T14:15:51.149021+0000","last_fullsized":"2024-02-13T14:15:51.149021+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:02:50.332899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150739+0000","last_change":"2024-02-13T14:15:23.620190+0000","last_active":"2024-02-13T14:15:51.150739+0000","last_peered":"2024-02-13T14:15:51.150739+0000","last_clean":"2024-02-13T14:15:51.150739+0000","last_became_active":"2024-02-13T14:15:23.619620+0000","last_became_peered":"2024-02-13T14:15:23.619620+0000","last_unstale":"2024-02-13T14:15:51.150739+0000","last_undegraded":"2024-02-13T14:15:51.150739+0000","last_fullsized":"2024-02-13T14:15:51.150739+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:44:41.067619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149853+0000","last_change":"2024-02-13T14:15:25.636616+0000","last_active":"2024-02-13T14:15:51.149853+0000","last_peered":"2024-02-13T14:15:51.149853+0000","last_clean":"2024-02-13T14:15:51.149853+0000","last_became_active":"2024-02-13T14:15:25.635951+0000","last_became_peered":"2024-02-13T14:15:25.635951+0000","last_unstale":"2024-02-13T14:15:51.149853+0000","last_undegraded":"2024-02-13T14:15:51.149853+0000","last_fullsized":"2024-02-13T14:15:51.149853+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:03:25.987871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148791+0000","last_change":"2024-02-13T14:15:29.648152+0000","last_active":"2024-02-13T14:15:51.148791+0000","last_peered":"2024-02-13T14:15:51.148791+0000","last_clean":"2024-02-13T14:15:51.148791+0000","last_became_active":"2024-02-13T14:15:29.647770+0000","last_became_peered":"2024-02-13T14:15:29.647770+0000","last_unstale":"2024-02-13T14:15:51.148791+0000","last_undegraded":"2024-02-13T14:15:51.148791+0000","last_fullsized":"2024-02-13T14:15:51.148791+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:44:47.872956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148877+0000","last_change":"2024-02-13T14:15:27.646752+0000","last_active":"2024-02-13T14:15:51.148877+0000","last_peered":"2024-02-13T14:15:51.148877+0000","last_clean":"2024-02-13T14:15:51.148877+0000","last_became_active":"2024-02-13T14:15:27.646401+0000","last_became_peered":"2024-02-13T14:15:27.646401+0000","last_unstale":"2024-02-13T14:15:51.148877+0000","last_undegraded":"2024-02-13T14:15:51.148877+0000","last_fullsized":"2024-02-13T14:15:51.148877+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:21:52.378118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148670+0000","last_change":"2024-02-13T14:15:23.621049+0000","last_active":"2024-02-13T14:15:51.148670+0000","last_peered":"2024-02-13T14:15:51.148670+0000","last_clean":"2024-02-13T14:15:51.148670+0000","last_became_active":"2024-02-13T14:15:23.620705+0000","last_became_peered":"2024-02-13T14:15:23.620705+0000","last_unstale":"2024-02-13T14:15:51.148670+0000","last_undegraded":"2024-02-13T14:15:51.148670+0000","last_fullsized":"2024-02-13T14:15:51.148670+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:50:45.636400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149054+0000","last_change":"2024-02-13T14:15:25.631891+0000","last_active":"2024-02-13T14:15:51.149054+0000","last_peered":"2024-02-13T14:15:51.149054+0000","last_clean":"2024-02-13T14:15:51.149054+0000","last_became_active":"2024-02-13T14:15:25.631483+0000","last_became_peered":"2024-02-13T14:15:25.631483+0000","last_unstale":"2024-02-13T14:15:51.149054+0000","last_undegraded":"2024-02-13T14:15:51.149054+0000","last_fullsized":"2024-02-13T14:15:51.149054+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:25:10.383502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149120+0000","last_change":"2024-02-13T14:15:29.644727+0000","last_active":"2024-02-13T14:15:51.149120+0000","last_peered":"2024-02-13T14:15:51.149120+0000","last_clean":"2024-02-13T14:15:51.149120+0000","last_became_active":"2024-02-13T14:15:29.644457+0000","last_became_peered":"2024-02-13T14:15:29.644457+0000","last_unstale":"2024-02-13T14:15:51.149120+0000","last_undegraded":"2024-02-13T14:15:51.149120+0000","last_fullsized":"2024-02-13T14:15:51.149120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:35:16.730471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150437+0000","last_change":"2024-02-13T14:15:23.616989+0000","last_active":"2024-02-13T14:15:51.150437+0000","last_peered":"2024-02-13T14:15:51.150437+0000","last_clean":"2024-02-13T14:15:51.150437+0000","last_became_active":"2024-02-13T14:15:23.616123+0000","last_became_peered":"2024-02-13T14:15:23.616123+0000","last_unstale":"2024-02-13T14:15:51.150437+0000","last_undegraded":"2024-02-13T14:15:51.150437+0000","last_fullsized":"2024-02-13T14:15:51.150437+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:18:04.651463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150047+0000","last_change":"2024-02-13T14:15:25.632523+0000","last_active":"2024-02-13T14:15:51.150047+0000","last_peered":"2024-02-13T14:15:51.150047+0000","last_clean":"2024-02-13T14:15:51.150047+0000","last_became_active":"2024-02-13T14:15:25.632205+0000","last_became_peered":"2024-02-13T14:15:25.632205+0000","last_unstale":"2024-02-13T14:15:51.150047+0000","last_undegraded":"2024-02-13T14:15:51.150047+0000","last_fullsized":"2024-02-13T14:15:51.150047+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:35:50.028643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669654+0000","last_change":"2024-02-13T14:15:27.641537+0000","last_active":"2024-02-13T14:16:03.669654+0000","last_peered":"2024-02-13T14:16:03.669654+0000","last_clean":"2024-02-13T14:16:03.669654+0000","last_became_active":"2024-02-13T14:15:27.640195+0000","last_became_peered":"2024-02-13T14:15:27.640195+0000","last_unstale":"2024-02-13T14:16:03.669654+0000","last_undegraded":"2024-02-13T14:16:03.669654+0000","last_fullsized":"2024-02-13T14:16:03.669654+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:36:53.918275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146573+0000","last_change":"2024-02-13T14:15:29.648143+0000","last_active":"2024-02-13T14:15:51.146573+0000","last_peered":"2024-02-13T14:15:51.146573+0000","last_clean":"2024-02-13T14:15:51.146573+0000","last_became_active":"2024-02-13T14:15:29.647699+0000","last_became_peered":"2024-02-13T14:15:29.647699+0000","last_unstale":"2024-02-13T14:15:51.146573+0000","last_undegraded":"2024-02-13T14:15:51.146573+0000","last_fullsized":"2024-02-13T14:15:51.146573+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:59:39.522434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148284+0000","last_change":"2024-02-13T14:15:23.621266+0000","last_active":"2024-02-13T14:15:51.148284+0000","last_peered":"2024-02-13T14:15:51.148284+0000","last_clean":"2024-02-13T14:15:51.148284+0000","last_became_active":"2024-02-13T14:15:23.620838+0000","last_became_peered":"2024-02-13T14:15:23.620838+0000","last_unstale":"2024-02-13T14:15:51.148284+0000","last_undegraded":"2024-02-13T14:15:51.148284+0000","last_fullsized":"2024-02-13T14:15:51.148284+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:11:15.677303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669578+0000","last_change":"2024-02-13T14:15:25.635009+0000","last_active":"2024-02-13T14:16:03.669578+0000","last_peered":"2024-02-13T14:16:03.669578+0000","last_clean":"2024-02-13T14:16:03.669578+0000","last_became_active":"2024-02-13T14:15:25.634395+0000","last_became_peered":"2024-02-13T14:15:25.634395+0000","last_unstale":"2024-02-13T14:16:03.669578+0000","last_undegraded":"2024-02-13T14:16:03.669578+0000","last_fullsized":"2024-02-13T14:16:03.669578+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:13:19.143472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148430+0000","last_change":"2024-02-13T14:15:27.643995+0000","last_active":"2024-02-13T14:15:51.148430+0000","last_peered":"2024-02-13T14:15:51.148430+0000","last_clean":"2024-02-13T14:15:51.148430+0000","last_became_active":"2024-02-13T14:15:27.640166+0000","last_became_peered":"2024-02-13T14:15:27.640166+0000","last_unstale":"2024-02-13T14:15:51.148430+0000","last_undegraded":"2024-02-13T14:15:51.148430+0000","last_fullsized":"2024-02-13T14:15:51.148430+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:07:32.813689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147845+0000","last_change":"2024-02-13T14:15:29.650567+0000","last_active":"2024-02-13T14:15:51.147845+0000","last_peered":"2024-02-13T14:15:51.147845+0000","last_clean":"2024-02-13T14:15:51.147845+0000","last_became_active":"2024-02-13T14:15:29.650245+0000","last_became_peered":"2024-02-13T14:15:29.650245+0000","last_unstale":"2024-02-13T14:15:51.147845+0000","last_undegraded":"2024-02-13T14:15:51.147845+0000","last_fullsized":"2024-02-13T14:15:51.147845+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:07:49.909687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152102+0000","last_change":"2024-02-13T14:15:23.622382+0000","last_active":"2024-02-13T14:15:51.152102+0000","last_peered":"2024-02-13T14:15:51.152102+0000","last_clean":"2024-02-13T14:15:51.152102+0000","last_became_active":"2024-02-13T14:15:23.621821+0000","last_became_peered":"2024-02-13T14:15:23.621821+0000","last_unstale":"2024-02-13T14:15:51.152102+0000","last_undegraded":"2024-02-13T14:15:51.152102+0000","last_fullsized":"2024-02-13T14:15:51.152102+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:34:21.313022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151323+0000","last_change":"2024-02-13T14:15:25.634245+0000","last_active":"2024-02-13T14:15:51.151323+0000","last_peered":"2024-02-13T14:15:51.151323+0000","last_clean":"2024-02-13T14:15:51.151323+0000","last_became_active":"2024-02-13T14:15:25.633731+0000","last_became_peered":"2024-02-13T14:15:25.633731+0000","last_unstale":"2024-02-13T14:15:51.151323+0000","last_undegraded":"2024-02-13T14:15:51.151323+0000","last_fullsized":"2024-02-13T14:15:51.151323+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:34:29.626079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147275+0000","last_change":"2024-02-13T14:15:27.644132+0000","last_active":"2024-02-13T14:15:51.147275+0000","last_peered":"2024-02-13T14:15:51.147275+0000","last_clean":"2024-02-13T14:15:51.147275+0000","last_became_active":"2024-02-13T14:15:27.643755+0000","last_became_peered":"2024-02-13T14:15:27.643755+0000","last_unstale":"2024-02-13T14:15:51.147275+0000","last_undegraded":"2024-02-13T14:15:51.147275+0000","last_fullsized":"2024-02-13T14:15:51.147275+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:23:51.227120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149521+0000","last_change":"2024-02-13T14:15:29.649490+0000","last_active":"2024-02-13T14:15:51.149521+0000","last_peered":"2024-02-13T14:15:51.149521+0000","last_clean":"2024-02-13T14:15:51.149521+0000","last_became_active":"2024-02-13T14:15:29.649110+0000","last_became_peered":"2024-02-13T14:15:29.649110+0000","last_unstale":"2024-02-13T14:15:51.149521+0000","last_undegraded":"2024-02-13T14:15:51.149521+0000","last_fullsized":"2024-02-13T14:15:51.149521+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:41:41.773389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147761+0000","last_change":"2024-02-13T14:15:25.632472+0000","last_active":"2024-02-13T14:15:51.147761+0000","last_peered":"2024-02-13T14:15:51.147761+0000","last_clean":"2024-02-13T14:15:51.147761+0000","last_became_active":"2024-02-13T14:15:25.632178+0000","last_became_peered":"2024-02-13T14:15:25.632178+0000","last_unstale":"2024-02-13T14:15:51.147761+0000","last_undegraded":"2024-02-13T14:15:51.147761+0000","last_fullsized":"2024-02-13T14:15:51.147761+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:37:54.057582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146657+0000","last_change":"2024-02-13T14:15:23.620438+0000","last_active":"2024-02-13T14:15:51.146657+0000","last_peered":"2024-02-13T14:15:51.146657+0000","last_clean":"2024-02-13T14:15:51.146657+0000","last_became_active":"2024-02-13T14:15:23.620125+0000","last_became_peered":"2024-02-13T14:15:23.620125+0000","last_unstale":"2024-02-13T14:15:51.146657+0000","last_undegraded":"2024-02-13T14:15:51.146657+0000","last_fullsized":"2024-02-13T14:15:51.146657+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:23:36.749528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149291+0000","last_change":"2024-02-13T14:15:27.645971+0000","last_active":"2024-02-13T14:15:51.149291+0000","last_peered":"2024-02-13T14:15:51.149291+0000","last_clean":"2024-02-13T14:15:51.149291+0000","last_became_active":"2024-02-13T14:15:27.645194+0000","last_became_peered":"2024-02-13T14:15:27.645194+0000","last_unstale":"2024-02-13T14:15:51.149291+0000","last_undegraded":"2024-02-13T14:15:51.149291+0000","last_fullsized":"2024-02-13T14:15:51.149291+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:56:04.877987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149338+0000","last_change":"2024-02-13T14:15:29.643780+0000","last_active":"2024-02-13T14:15:51.149338+0000","last_peered":"2024-02-13T14:15:51.149338+0000","last_clean":"2024-02-13T14:15:51.149338+0000","last_became_active":"2024-02-13T14:15:29.643523+0000","last_became_peered":"2024-02-13T14:15:29.643523+0000","last_unstale":"2024-02-13T14:15:51.149338+0000","last_undegraded":"2024-02-13T14:15:51.149338+0000","last_fullsized":"2024-02-13T14:15:51.149338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:47:24.677905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146319+0000","last_change":"2024-02-13T14:15:25.642347+0000","last_active":"2024-02-13T14:15:51.146319+0000","last_peered":"2024-02-13T14:15:51.146319+0000","last_clean":"2024-02-13T14:15:51.146319+0000","last_became_active":"2024-02-13T14:15:25.642004+0000","last_became_peered":"2024-02-13T14:15:25.642004+0000","last_unstale":"2024-02-13T14:15:51.146319+0000","last_undegraded":"2024-02-13T14:15:51.146319+0000","last_fullsized":"2024-02-13T14:15:51.146319+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:08:39.160948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150783+0000","last_change":"2024-02-13T14:15:23.619762+0000","last_active":"2024-02-13T14:15:51.150783+0000","last_peered":"2024-02-13T14:15:51.150783+0000","last_clean":"2024-02-13T14:15:51.150783+0000","last_became_active":"2024-02-13T14:15:23.619370+0000","last_became_peered":"2024-02-13T14:15:23.619370+0000","last_unstale":"2024-02-13T14:15:51.150783+0000","last_undegraded":"2024-02-13T14:15:51.150783+0000","last_fullsized":"2024-02-13T14:15:51.150783+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:10:07.714428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:53.249303+0000","last_change":"2024-02-13T14:15:13.265703+0000","last_active":"2024-02-13T14:15:53.249303+0000","last_peered":"2024-02-13T14:15:53.249303+0000","last_clean":"2024-02-13T14:15:53.249303+0000","last_became_active":"2024-02-13T14:15:13.265341+0000","last_became_peered":"2024-02-13T14:15:13.265341+0000","last_unstale":"2024-02-13T14:15:53.249303+0000","last_undegraded":"2024-02-13T14:15:53.249303+0000","last_fullsized":"2024-02-13T14:15:53.249303+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:12.252871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:12.252871+0000","last_clean_scrub_stamp":"2024-02-13T14:15:12.252871+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:28:21.863788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152204+0000","last_change":"2024-02-13T14:15:27.639785+0000","last_active":"2024-02-13T14:15:51.152204+0000","last_peered":"2024-02-13T14:15:51.152204+0000","last_clean":"2024-02-13T14:15:51.152204+0000","last_became_active":"2024-02-13T14:15:27.639258+0000","last_became_peered":"2024-02-13T14:15:27.639258+0000","last_unstale":"2024-02-13T14:15:51.152204+0000","last_undegraded":"2024-02-13T14:15:51.152204+0000","last_fullsized":"2024-02-13T14:15:51.152204+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:57:34.751415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151350+0000","last_change":"2024-02-13T14:15:29.646852+0000","last_active":"2024-02-13T14:15:51.151350+0000","last_peered":"2024-02-13T14:15:51.151350+0000","last_clean":"2024-02-13T14:15:51.151350+0000","last_became_active":"2024-02-13T14:15:29.645962+0000","last_became_peered":"2024-02-13T14:15:29.645962+0000","last_unstale":"2024-02-13T14:15:51.151350+0000","last_undegraded":"2024-02-13T14:15:51.151350+0000","last_fullsized":"2024-02-13T14:15:51.151350+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:04:26.660676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146944+0000","last_change":"2024-02-13T14:15:25.638447+0000","last_active":"2024-02-13T14:15:51.146944+0000","last_peered":"2024-02-13T14:15:51.146944+0000","last_clean":"2024-02-13T14:15:51.146944+0000","last_became_active":"2024-02-13T14:15:25.638083+0000","last_became_peered":"2024-02-13T14:15:25.638083+0000","last_unstale":"2024-02-13T14:15:51.146944+0000","last_undegraded":"2024-02-13T14:15:51.146944+0000","last_fullsized":"2024-02-13T14:15:51.146944+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:05:02.719912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149788+0000","last_change":"2024-02-13T14:15:23.621788+0000","last_active":"2024-02-13T14:15:51.149788+0000","last_peered":"2024-02-13T14:15:51.149788+0000","last_clean":"2024-02-13T14:15:51.149788+0000","last_became_active":"2024-02-13T14:15:23.621424+0000","last_became_peered":"2024-02-13T14:15:23.621424+0000","last_unstale":"2024-02-13T14:15:51.149788+0000","last_undegraded":"2024-02-13T14:15:51.149788+0000","last_fullsized":"2024-02-13T14:15:51.149788+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:42:51.394977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149770+0000","last_change":"2024-02-13T14:15:27.643151+0000","last_active":"2024-02-13T14:15:51.149770+0000","last_peered":"2024-02-13T14:15:51.149770+0000","last_clean":"2024-02-13T14:15:51.149770+0000","last_became_active":"2024-02-13T14:15:27.642859+0000","last_became_peered":"2024-02-13T14:15:27.642859+0000","last_unstale":"2024-02-13T14:15:51.149770+0000","last_undegraded":"2024-02-13T14:15:51.149770+0000","last_fullsized":"2024-02-13T14:15:51.149770+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:09:50.204819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151854+0000","last_change":"2024-02-13T14:15:29.646953+0000","last_active":"2024-02-13T14:15:51.151854+0000","last_peered":"2024-02-13T14:15:51.151854+0000","last_clean":"2024-02-13T14:15:51.151854+0000","last_became_active":"2024-02-13T14:15:29.646488+0000","last_became_peered":"2024-02-13T14:15:29.646488+0000","last_unstale":"2024-02-13T14:15:51.151854+0000","last_undegraded":"2024-02-13T14:15:51.151854+0000","last_fullsized":"2024-02-13T14:15:51.151854+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:44:13.181991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150132+0000","last_change":"2024-02-13T14:15:23.618522+0000","last_active":"2024-02-13T14:15:51.150132+0000","last_peered":"2024-02-13T14:15:51.150132+0000","last_clean":"2024-02-13T14:15:51.150132+0000","last_became_active":"2024-02-13T14:15:23.617267+0000","last_became_peered":"2024-02-13T14:15:23.617267+0000","last_unstale":"2024-02-13T14:15:51.150132+0000","last_undegraded":"2024-02-13T14:15:51.150132+0000","last_fullsized":"2024-02-13T14:15:51.150132+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:14:37.593450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":81,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669401+0000","last_change":"2024-02-13T14:15:25.641222+0000","last_active":"2024-02-13T14:16:03.669401+0000","last_peered":"2024-02-13T14:16:03.669401+0000","last_clean":"2024-02-13T14:16:03.669401+0000","last_became_active":"2024-02-13T14:15:25.640878+0000","last_became_peered":"2024-02-13T14:15:25.640878+0000","last_unstale":"2024-02-13T14:16:03.669401+0000","last_undegraded":"2024-02-13T14:16:03.669401+0000","last_fullsized":"2024-02-13T14:16:03.669401+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:45:47.004260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669662+0000","last_change":"2024-02-13T14:15:27.641270+0000","last_active":"2024-02-13T14:16:03.669662+0000","last_peered":"2024-02-13T14:16:03.669662+0000","last_clean":"2024-02-13T14:16:03.669662+0000","last_became_active":"2024-02-13T14:15:27.639999+0000","last_became_peered":"2024-02-13T14:15:27.639999+0000","last_unstale":"2024-02-13T14:16:03.669662+0000","last_undegraded":"2024-02-13T14:16:03.669662+0000","last_fullsized":"2024-02-13T14:16:03.669662+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:56:08.559879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149782+0000","last_change":"2024-02-13T14:15:29.644978+0000","last_active":"2024-02-13T14:15:51.149782+0000","last_peered":"2024-02-13T14:15:51.149782+0000","last_clean":"2024-02-13T14:15:51.149782+0000","last_became_active":"2024-02-13T14:15:29.644628+0000","last_became_peered":"2024-02-13T14:15:29.644628+0000","last_unstale":"2024-02-13T14:15:51.149782+0000","last_undegraded":"2024-02-13T14:15:51.149782+0000","last_fullsized":"2024-02-13T14:15:51.149782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:16:11.551738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'2","reported_seq":34,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149049+0000","last_change":"2024-02-13T14:15:23.622189+0000","last_active":"2024-02-13T14:15:51.149049+0000","last_peered":"2024-02-13T14:15:51.149049+0000","last_clean":"2024-02-13T14:15:51.149049+0000","last_became_active":"2024-02-13T14:15:23.621872+0000","last_became_peered":"2024-02-13T14:15:23.621872+0000","last_unstale":"2024-02-13T14:15:51.149049+0000","last_undegraded":"2024-02-13T14:15:51.149049+0000","last_fullsized":"2024-02-13T14:15:51.149049+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:30:25.065018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":920,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149858+0000","last_change":"2024-02-13T14:15:25.638472+0000","last_active":"2024-02-13T14:15:51.149858+0000","last_peered":"2024-02-13T14:15:51.149858+0000","last_clean":"2024-02-13T14:15:51.149858+0000","last_became_active":"2024-02-13T14:15:25.638020+0000","last_became_peered":"2024-02-13T14:15:25.638020+0000","last_unstale":"2024-02-13T14:15:51.149858+0000","last_undegraded":"2024-02-13T14:15:51.149858+0000","last_fullsized":"2024-02-13T14:15:51.149858+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:53:47.502835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669407+0000","last_change":"2024-02-13T14:15:27.642826+0000","last_active":"2024-02-13T14:16:03.669407+0000","last_peered":"2024-02-13T14:16:03.669407+0000","last_clean":"2024-02-13T14:16:03.669407+0000","last_became_active":"2024-02-13T14:15:27.642512+0000","last_became_peered":"2024-02-13T14:15:27.642512+0000","last_unstale":"2024-02-13T14:16:03.669407+0000","last_undegraded":"2024-02-13T14:16:03.669407+0000","last_fullsized":"2024-02-13T14:16:03.669407+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:59:13.384374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149116+0000","last_change":"2024-02-13T14:15:29.644943+0000","last_active":"2024-02-13T14:15:51.149116+0000","last_peered":"2024-02-13T14:15:51.149116+0000","last_clean":"2024-02-13T14:15:51.149116+0000","last_became_active":"2024-02-13T14:15:29.644658+0000","last_became_peered":"2024-02-13T14:15:29.644658+0000","last_unstale":"2024-02-13T14:15:51.149116+0000","last_undegraded":"2024-02-13T14:15:51.149116+0000","last_fullsized":"2024-02-13T14:15:51.149116+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:07.119765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151834+0000","last_change":"2024-02-13T14:15:23.621272+0000","last_active":"2024-02-13T14:15:51.151834+0000","last_peered":"2024-02-13T14:15:51.151834+0000","last_clean":"2024-02-13T14:15:51.151834+0000","last_became_active":"2024-02-13T14:15:23.620828+0000","last_became_peered":"2024-02-13T14:15:23.620828+0000","last_unstale":"2024-02-13T14:15:51.151834+0000","last_undegraded":"2024-02-13T14:15:51.151834+0000","last_fullsized":"2024-02-13T14:15:51.151834+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:55:36.249591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149378+0000","last_change":"2024-02-13T14:15:25.641422+0000","last_active":"2024-02-13T14:15:51.149378+0000","last_peered":"2024-02-13T14:15:51.149378+0000","last_clean":"2024-02-13T14:15:51.149378+0000","last_became_active":"2024-02-13T14:15:25.640836+0000","last_became_peered":"2024-02-13T14:15:25.640836+0000","last_unstale":"2024-02-13T14:15:51.149378+0000","last_undegraded":"2024-02-13T14:15:51.149378+0000","last_fullsized":"2024-02-13T14:15:51.149378+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:30:26.719450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148145+0000","last_change":"2024-02-13T14:15:27.642548+0000","last_active":"2024-02-13T14:15:51.148145+0000","last_peered":"2024-02-13T14:15:51.148145+0000","last_clean":"2024-02-13T14:15:51.148145+0000","last_became_active":"2024-02-13T14:15:27.642134+0000","last_became_peered":"2024-02-13T14:15:27.642134+0000","last_unstale":"2024-02-13T14:15:51.148145+0000","last_undegraded":"2024-02-13T14:15:51.148145+0000","last_fullsized":"2024-02-13T14:15:51.148145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T02:12:09.865933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151489+0000","last_change":"2024-02-13T14:15:29.647395+0000","last_active":"2024-02-13T14:15:51.151489+0000","last_peered":"2024-02-13T14:15:51.151489+0000","last_clean":"2024-02-13T14:15:51.151489+0000","last_became_active":"2024-02-13T14:15:29.646818+0000","last_became_peered":"2024-02-13T14:15:29.646818+0000","last_unstale":"2024-02-13T14:15:51.151489+0000","last_undegraded":"2024-02-13T14:15:51.151489+0000","last_fullsized":"2024-02-13T14:15:51.151489+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:00:29.235602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":37,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669480+0000","last_change":"2024-02-13T14:15:27.640960+0000","last_active":"2024-02-13T14:16:03.669480+0000","last_peered":"2024-02-13T14:16:03.669480+0000","last_clean":"2024-02-13T14:16:03.669480+0000","last_became_active":"2024-02-13T14:15:27.640521+0000","last_became_peered":"2024-02-13T14:15:27.640521+0000","last_unstale":"2024-02-13T14:16:03.669480+0000","last_undegraded":"2024-02-13T14:16:03.669480+0000","last_fullsized":"2024-02-13T14:16:03.669480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:21:08.223607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149649+0000","last_change":"2024-02-13T14:15:23.619792+0000","last_active":"2024-02-13T14:15:51.149649+0000","last_peered":"2024-02-13T14:15:51.149649+0000","last_clean":"2024-02-13T14:15:51.149649+0000","last_became_active":"2024-02-13T14:15:23.619485+0000","last_became_peered":"2024-02-13T14:15:23.619485+0000","last_unstale":"2024-02-13T14:15:51.149649+0000","last_undegraded":"2024-02-13T14:15:51.149649+0000","last_fullsized":"2024-02-13T14:15:51.149649+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:32:43.743066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149263+0000","last_change":"2024-02-13T14:15:25.631908+0000","last_active":"2024-02-13T14:15:51.149263+0000","last_peered":"2024-02-13T14:15:51.149263+0000","last_clean":"2024-02-13T14:15:51.149263+0000","last_became_active":"2024-02-13T14:15:25.631547+0000","last_became_peered":"2024-02-13T14:15:25.631547+0000","last_unstale":"2024-02-13T14:15:51.149263+0000","last_undegraded":"2024-02-13T14:15:51.149263+0000","last_fullsized":"2024-02-13T14:15:51.149263+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:02:31.023771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150223+0000","last_change":"2024-02-13T14:15:29.644991+0000","last_active":"2024-02-13T14:15:51.150223+0000","last_peered":"2024-02-13T14:15:51.150223+0000","last_clean":"2024-02-13T14:15:51.150223+0000","last_became_active":"2024-02-13T14:15:29.644689+0000","last_became_peered":"2024-02-13T14:15:29.644689+0000","last_unstale":"2024-02-13T14:15:51.150223+0000","last_undegraded":"2024-02-13T14:15:51.150223+0000","last_fullsized":"2024-02-13T14:15:51.150223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T14:56:03.551408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149988+0000","last_change":"2024-02-13T14:15:27.645507+0000","last_active":"2024-02-13T14:15:51.149988+0000","last_peered":"2024-02-13T14:15:51.149988+0000","last_clean":"2024-02-13T14:15:51.149988+0000","last_became_active":"2024-02-13T14:15:27.644436+0000","last_became_peered":"2024-02-13T14:15:27.644436+0000","last_unstale":"2024-02-13T14:15:51.149988+0000","last_undegraded":"2024-02-13T14:15:51.149988+0000","last_fullsized":"2024-02-13T14:15:51.149988+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T01:24:30.862107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146762+0000","last_change":"2024-02-13T14:15:23.621592+0000","last_active":"2024-02-13T14:15:51.146762+0000","last_peered":"2024-02-13T14:15:51.146762+0000","last_clean":"2024-02-13T14:15:51.146762+0000","last_became_active":"2024-02-13T14:15:23.621200+0000","last_became_peered":"2024-02-13T14:15:23.621200+0000","last_unstale":"2024-02-13T14:15:51.146762+0000","last_undegraded":"2024-02-13T14:15:51.146762+0000","last_fullsized":"2024-02-13T14:15:51.146762+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:25:21.143290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148705+0000","last_change":"2024-02-13T14:15:25.632253+0000","last_active":"2024-02-13T14:15:51.148705+0000","last_peered":"2024-02-13T14:15:51.148705+0000","last_clean":"2024-02-13T14:15:51.148705+0000","last_became_active":"2024-02-13T14:15:25.631995+0000","last_became_peered":"2024-02-13T14:15:25.631995+0000","last_unstale":"2024-02-13T14:15:51.148705+0000","last_undegraded":"2024-02-13T14:15:51.148705+0000","last_fullsized":"2024-02-13T14:15:51.148705+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:34:53.032935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149678+0000","last_change":"2024-02-13T14:15:29.643765+0000","last_active":"2024-02-13T14:15:51.149678+0000","last_peered":"2024-02-13T14:15:51.149678+0000","last_clean":"2024-02-13T14:15:51.149678+0000","last_became_active":"2024-02-13T14:15:29.643480+0000","last_became_peered":"2024-02-13T14:15:29.643480+0000","last_unstale":"2024-02-13T14:15:51.149678+0000","last_undegraded":"2024-02-13T14:15:51.149678+0000","last_fullsized":"2024-02-13T14:15:51.149678+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:28:04.900711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148896+0000","last_change":"2024-02-13T14:15:23.616944+0000","last_active":"2024-02-13T14:15:51.148896+0000","last_peered":"2024-02-13T14:15:51.148896+0000","last_clean":"2024-02-13T14:15:51.148896+0000","last_became_active":"2024-02-13T14:15:23.616074+0000","last_became_peered":"2024-02-13T14:15:23.616074+0000","last_unstale":"2024-02-13T14:15:51.148896+0000","last_undegraded":"2024-02-13T14:15:51.148896+0000","last_fullsized":"2024-02-13T14:15:51.148896+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:39:33.605890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147042+0000","last_change":"2024-02-13T14:15:25.638441+0000","last_active":"2024-02-13T14:15:51.147042+0000","last_peered":"2024-02-13T14:15:51.147042+0000","last_clean":"2024-02-13T14:15:51.147042+0000","last_became_active":"2024-02-13T14:15:25.637846+0000","last_became_peered":"2024-02-13T14:15:25.637846+0000","last_unstale":"2024-02-13T14:15:51.147042+0000","last_undegraded":"2024-02-13T14:15:51.147042+0000","last_fullsized":"2024-02-13T14:15:51.147042+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:02:49.262286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150985+0000","last_change":"2024-02-13T14:15:27.640026+0000","last_active":"2024-02-13T14:15:51.150985+0000","last_peered":"2024-02-13T14:15:51.150985+0000","last_clean":"2024-02-13T14:15:51.150985+0000","last_became_active":"2024-02-13T14:15:27.639157+0000","last_became_peered":"2024-02-13T14:15:27.639157+0000","last_unstale":"2024-02-13T14:15:51.150985+0000","last_undegraded":"2024-02-13T14:15:51.150985+0000","last_fullsized":"2024-02-13T14:15:51.150985+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T20:35:42.139535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.147114+0000","last_change":"2024-02-13T14:15:29.648975+0000","last_active":"2024-02-13T14:15:51.147114+0000","last_peered":"2024-02-13T14:15:51.147114+0000","last_clean":"2024-02-13T14:15:51.147114+0000","last_became_active":"2024-02-13T14:15:29.648559+0000","last_became_peered":"2024-02-13T14:15:29.648559+0000","last_unstale":"2024-02-13T14:15:51.147114+0000","last_undegraded":"2024-02-13T14:15:51.147114+0000","last_fullsized":"2024-02-13T14:15:51.147114+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T23:22:12.380243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":40,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:16:03.669490+0000","last_change":"2024-02-13T14:15:27.640994+0000","last_active":"2024-02-13T14:16:03.669490+0000","last_peered":"2024-02-13T14:16:03.669490+0000","last_clean":"2024-02-13T14:16:03.669490+0000","last_became_active":"2024-02-13T14:15:27.640585+0000","last_became_peered":"2024-02-13T14:15:27.640585+0000","last_unstale":"2024-02-13T14:16:03.669490+0000","last_undegraded":"2024-02-13T14:16:03.669490+0000","last_fullsized":"2024-02-13T14:16:03.669490+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:28:07.893873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149119+0000","last_change":"2024-02-13T14:15:23.620047+0000","last_active":"2024-02-13T14:15:51.149119+0000","last_peered":"2024-02-13T14:15:51.149119+0000","last_clean":"2024-02-13T14:15:51.149119+0000","last_became_active":"2024-02-13T14:15:23.618628+0000","last_became_peered":"2024-02-13T14:15:23.618628+0000","last_unstale":"2024-02-13T14:15:51.149119+0000","last_undegraded":"2024-02-13T14:15:51.149119+0000","last_fullsized":"2024-02-13T14:15:51.149119+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:23:16.406845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152270+0000","last_change":"2024-02-13T14:15:25.637588+0000","last_active":"2024-02-13T14:15:51.152270+0000","last_peered":"2024-02-13T14:15:51.152270+0000","last_clean":"2024-02-13T14:15:51.152270+0000","last_became_active":"2024-02-13T14:15:25.637034+0000","last_became_peered":"2024-02-13T14:15:25.637034+0000","last_unstale":"2024-02-13T14:15:51.152270+0000","last_undegraded":"2024-02-13T14:15:51.152270+0000","last_fullsized":"2024-02-13T14:15:51.152270+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:20:24.283562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148515+0000","last_change":"2024-02-13T14:15:29.643954+0000","last_active":"2024-02-13T14:15:51.148515+0000","last_peered":"2024-02-13T14:15:51.148515+0000","last_clean":"2024-02-13T14:15:51.148515+0000","last_became_active":"2024-02-13T14:15:29.643664+0000","last_became_peered":"2024-02-13T14:15:29.643664+0000","last_unstale":"2024-02-13T14:15:51.148515+0000","last_undegraded":"2024-02-13T14:15:51.148515+0000","last_fullsized":"2024-02-13T14:15:51.148515+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-15T00:20:23.578065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.150744+0000","last_change":"2024-02-13T14:15:23.618511+0000","last_active":"2024-02-13T14:15:51.150744+0000","last_peered":"2024-02-13T14:15:51.150744+0000","last_clean":"2024-02-13T14:15:51.150744+0000","last_became_active":"2024-02-13T14:15:23.617385+0000","last_became_peered":"2024-02-13T14:15:23.617385+0000","last_unstale":"2024-02-13T14:15:51.150744+0000","last_undegraded":"2024-02-13T14:15:51.150744+0000","last_fullsized":"2024-02-13T14:15:51.150744+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T21:34:51.554706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.152432+0000","last_change":"2024-02-13T14:15:25.633129+0000","last_active":"2024-02-13T14:15:51.152432+0000","last_peered":"2024-02-13T14:15:51.152432+0000","last_clean":"2024-02-13T14:15:51.152432+0000","last_became_active":"2024-02-13T14:15:25.632752+0000","last_became_peered":"2024-02-13T14:15:25.632752+0000","last_unstale":"2024-02-13T14:15:51.152432+0000","last_undegraded":"2024-02-13T14:15:51.152432+0000","last_fullsized":"2024-02-13T14:15:51.152432+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:17:38.138091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151691+0000","last_change":"2024-02-13T14:15:27.641118+0000","last_active":"2024-02-13T14:15:51.151691+0000","last_peered":"2024-02-13T14:15:51.151691+0000","last_clean":"2024-02-13T14:15:51.151691+0000","last_became_active":"2024-02-13T14:15:27.640730+0000","last_became_peered":"2024-02-13T14:15:27.640730+0000","last_unstale":"2024-02-13T14:15:51.151691+0000","last_undegraded":"2024-02-13T14:15:51.151691+0000","last_fullsized":"2024-02-13T14:15:51.151691+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:16:17.826352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.145942+0000","last_change":"2024-02-13T14:15:29.648133+0000","last_active":"2024-02-13T14:15:51.145942+0000","last_peered":"2024-02-13T14:15:51.145942+0000","last_clean":"2024-02-13T14:15:51.145942+0000","last_became_active":"2024-02-13T14:15:29.647724+0000","last_became_peered":"2024-02-13T14:15:29.647724+0000","last_unstale":"2024-02-13T14:15:51.145942+0000","last_undegraded":"2024-02-13T14:15:51.145942+0000","last_fullsized":"2024-02-13T14:15:51.145942+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T22:23:38.513000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149936+0000","last_change":"2024-02-13T14:15:23.620598+0000","last_active":"2024-02-13T14:15:51.149936+0000","last_peered":"2024-02-13T14:15:51.149936+0000","last_clean":"2024-02-13T14:15:51.149936+0000","last_became_active":"2024-02-13T14:15:23.619680+0000","last_became_peered":"2024-02-13T14:15:23.619680+0000","last_unstale":"2024-02-13T14:15:51.149936+0000","last_undegraded":"2024-02-13T14:15:51.149936+0000","last_fullsized":"2024-02-13T14:15:51.149936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:35:39.930620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151309+0000","last_change":"2024-02-13T14:15:25.636342+0000","last_active":"2024-02-13T14:15:51.151309+0000","last_peered":"2024-02-13T14:15:51.151309+0000","last_clean":"2024-02-13T14:15:51.151309+0000","last_became_active":"2024-02-13T14:15:25.635863+0000","last_became_peered":"2024-02-13T14:15:25.635863+0000","last_unstale":"2024-02-13T14:15:51.151309+0000","last_undegraded":"2024-02-13T14:15:51.151309+0000","last_fullsized":"2024-02-13T14:15:51.151309+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:52:35.698341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.146723+0000","last_change":"2024-02-13T14:15:27.646394+0000","last_active":"2024-02-13T14:15:51.146723+0000","last_peered":"2024-02-13T14:15:51.146723+0000","last_clean":"2024-02-13T14:15:51.146723+0000","last_became_active":"2024-02-13T14:15:27.646020+0000","last_became_peered":"2024-02-13T14:15:27.646020+0000","last_unstale":"2024-02-13T14:15:51.146723+0000","last_undegraded":"2024-02-13T14:15:51.146723+0000","last_fullsized":"2024-02-13T14:15:51.146723+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T16:05:00.316500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149611+0000","last_change":"2024-02-13T14:15:29.646741+0000","last_active":"2024-02-13T14:15:51.149611+0000","last_peered":"2024-02-13T14:15:51.149611+0000","last_clean":"2024-02-13T14:15:51.149611+0000","last_became_active":"2024-02-13T14:15:29.646281+0000","last_became_peered":"2024-02-13T14:15:29.646281+0000","last_unstale":"2024-02-13T14:15:51.149611+0000","last_undegraded":"2024-02-13T14:15:51.149611+0000","last_fullsized":"2024-02-13T14:15:51.149611+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:28.628444+0000","last_clean_scrub_stamp":"2024-02-13T14:15:28.628444+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T17:50:10.252404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.149279+0000","last_change":"2024-02-13T14:15:23.623850+0000","last_active":"2024-02-13T14:15:51.149279+0000","last_peered":"2024-02-13T14:15:51.149279+0000","last_clean":"2024-02-13T14:15:51.149279+0000","last_became_active":"2024-02-13T14:15:23.623544+0000","last_became_peered":"2024-02-13T14:15:23.623544+0000","last_unstale":"2024-02-13T14:15:51.149279+0000","last_undegraded":"2024-02-13T14:15:51.149279+0000","last_fullsized":"2024-02-13T14:15:51.149279+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:22.602588+0000","last_clean_scrub_stamp":"2024-02-13T14:15:22.602588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T15:38:14.058405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.151935+0000","last_change":"2024-02-13T14:15:25.636494+0000","last_active":"2024-02-13T14:15:51.151935+0000","last_peered":"2024-02-13T14:15:51.151935+0000","last_clean":"2024-02-13T14:15:51.151935+0000","last_became_active":"2024-02-13T14:15:25.636113+0000","last_became_peered":"2024-02-13T14:15:25.636113+0000","last_unstale":"2024-02-13T14:15:51.151935+0000","last_undegraded":"2024-02-13T14:15:51.151935+0000","last_fullsized":"2024-02-13T14:15:51.151935+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:24.615840+0000","last_clean_scrub_stamp":"2024-02-13T14:15:24.615840+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T19:56:12.278547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-13T14:15:51.148886+0000","last_change":"2024-02-13T14:15:27.639612+0000","last_active":"2024-02-13T14:15:51.148886+0000","last_peered":"2024-02-13T14:15:51.148886+0000","last_clean":"2024-02-13T14:15:51.148886+0000","last_became_active":"2024-02-13T14:15:27.639326+0000","last_became_peered":"2024-02-13T14:15:27.639326+0000","last_unstale":"2024-02-13T14:15:51.148886+0000","last_undegraded":"2024-02-13T14:15:51.148886+0000","last_fullsized":"2024-02-13T14:15:51.148886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-13T14:15:26.624492+0000","last_clean_scrub_stamp":"2024-02-13T14:15:26.624492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-14T18:51:52.724176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411343,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738393,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Feb 13 14:15:47 2024","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.38300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Feb 13 14:15:49 2024","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.17100000000000001,"5min":0.17100000000000001,"15min":0.17100000000000001},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.40400000000000003}]},{"osd":2,"last update":"Tue Feb 13 14:16:11 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"last":0.501},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.54900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-13T14:16:14.753 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-13T14:16:14.753 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-13T14:16:14.753 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-13T14:16:14.753 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- ceph health --format=json 2024-02-13T14:16:14.998 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:15.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:15 smithi170 ceph-mon[30181]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:15.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:15 smithi170 ceph-mon[30181]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:15.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:15 smithi170 ceph-mon[30181]: Deploying daemon grafana.a on smithi078 2024-02-13T14:16:15.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:15 smithi078 ceph-mon[30086]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:15.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:15 smithi078 ceph-mon[30086]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:15.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:15 smithi078 ceph-mon[30086]: Deploying daemon grafana.a on smithi078 2024-02-13T14:16:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:15 smithi055 ceph-mon[25769]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-13T14:16:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:15 smithi055 ceph-mon[25769]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-13T14:16:15.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:15 smithi055 ceph-mon[25769]: Deploying daemon grafana.a on smithi078 2024-02-13T14:16:16.144 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:16 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:16.045Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000634316s 2024-02-13T14:16:16.461 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:16 smithi055 ceph-mon[25769]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:16.461 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:16 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:16.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:16 smithi170 ceph-mon[30181]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:16.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:16 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:16.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:16 smithi078 ceph-mon[30086]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:16.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:16 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:16.951 INFO:teuthology.orchestra.run.smithi055.stdout: 2024-02-13T14:16:16.951 INFO:teuthology.orchestra.run.smithi055.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-13T14:16:17.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:17 smithi055 ceph-mon[25769]: from='client.? 172.21.15.55:0/1624974023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:17.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:17 smithi170 ceph-mon[30181]: from='client.? 172.21.15.55:0/1624974023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:17.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:17 smithi078 ceph-mon[30086]: from='client.? 172.21.15.55:0/1624974023' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-13T14:16:17.865 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-13T14:16:17.865 INFO:tasks.cephadm:Setup complete, yielding 2024-02-13T14:16:17.865 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-13T14:16:17.880 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi055.front.sepia.ceph.com 2024-02-13T14:16:17.882 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-13T14:16:18.132 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:18.512 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:18 smithi055 ceph-mon[25769]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:18.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:18 smithi170 ceph-mon[30181]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:18.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:18 smithi078 ceph-mon[30086]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:19.727 INFO:teuthology.orchestra.run.smithi055.stdout:167 167 2024-02-13T14:16:20.461 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch status' 2024-02-13T14:16:20.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:20 smithi170 ceph-mon[30181]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:20.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:20 smithi078 ceph-mon[30086]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:20.705 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:20.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:20 smithi055 ceph-mon[25769]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:21.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:21 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:21.815 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:21 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:21.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:21 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:Backend: cephadm 2024-02-13T14:16:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:Available: Yes 2024-02-13T14:16:22.589 INFO:teuthology.orchestra.run.smithi055.stdout:Paused: No 2024-02-13T14:16:22.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:22 smithi078 ceph-mon[30086]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:22.814 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:22 smithi055 ceph-mon[25769]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:22.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:22 smithi170 ceph-mon[30181]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:23.252 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:23 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:16:23] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:23.286 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-13T14:16:23.532 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:23.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:23 smithi055 ceph-mon[25769]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:23.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:23 smithi078 ceph-mon[30086]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:23.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:23 smithi170 ceph-mon[30181]: from='client.24395 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:24.315 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:16:24 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:16:24.048Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003614604s 2024-02-13T14:16:24.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:24 smithi078 ceph-mon[30086]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:24.716 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:24 smithi055 ceph-mon[25769]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:24.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:24 smithi170 ceph-mon[30181]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:alertmanager.a smithi055 *:9093,9094 starting - - - - 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.x smithi078 *:8443,8765 running (2m) 31s ago 2m 447M - 19.0.0-1423-ge1b79d5a 28480620e5ed cab1c8d4cff1 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.y smithi055 *:9283,8765 running (4m) 25s ago 4m 481M - 19.0.0-1423-ge1b79d5a 28480620e5ed 754d50c91a1d 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mgr.z smithi170 *:8443,8765 running (2m) 32s ago 2m 447M - 19.0.0-1423-ge1b79d5a 28480620e5ed a56913adfd51 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mon.a smithi055 running (4m) 25s ago 4m 50.4M 2048M 19.0.0-1423-ge1b79d5a 28480620e5ed 673ad2dec541 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mon.b smithi078 running (2m) 31s ago 2m 41.7M 2048M 19.0.0-1423-ge1b79d5a 28480620e5ed 37fae4a395b5 2024-02-13T14:16:25.457 INFO:teuthology.orchestra.run.smithi055.stdout:mon.c smithi170 running (2m) 32s ago 2m 45.4M 2048M 19.0.0-1423-ge1b79d5a 28480620e5ed a50dafabb547 2024-02-13T14:16:25.458 INFO:teuthology.orchestra.run.smithi055.stdout:osd.0 smithi055 running (2m) 25s ago 2m 59.4M 15.6G 19.0.0-1423-ge1b79d5a 28480620e5ed 7eea229474c3 2024-02-13T14:16:25.458 INFO:teuthology.orchestra.run.smithi055.stdout:osd.1 smithi078 running (105s) 31s ago 104s 58.1M 15.6G 19.0.0-1423-ge1b79d5a 28480620e5ed a2d6fbd6047c 2024-02-13T14:16:25.458 INFO:teuthology.orchestra.run.smithi055.stdout:osd.2 smithi170 running (81s) 32s ago 81s 61.4M 16.6G 19.0.0-1423-ge1b79d5a 28480620e5ed d49a9f147f6d 2024-02-13T14:16:25.458 INFO:teuthology.orchestra.run.smithi055.stdout:prometheus.a smithi078 *:9095 running (43s) 31s ago 42s 24.2M - 2.43.0 a07b618ecd1d 91836debb8c6 2024-02-13T14:16:25.458 INFO:teuthology.orchestra.run.smithi055.stdout:rgw.foo.a smithi055 *:80 running (63s) 25s ago 63s 76.1M - 19.0.0-1423-ge1b79d5a 28480620e5ed b9a0e6009978 2024-02-13T14:16:26.111 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-13T14:16:26.350 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:26.693 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:26 smithi055 ceph-mon[25769]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:26.694 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:26 smithi055 ceph-mon[25769]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:26.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:26 smithi078 ceph-mon[30086]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:26.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:26 smithi078 ceph-mon[30086]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:26.709 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:26 smithi170 ceph-mon[30181]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:26.709 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:26 smithi170 ceph-mon[30181]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:alertmanager ?:9093,9094 0/1 - 61s smithi055=a;count:1 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:grafana ?:3000 0/1 - 58s smithi078=a;count:1 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:mgr 3/3 35s ago 2m smithi055=y;smithi078=x;smithi170=z;count:3 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:mon 3/3 35s ago 3m smithi055:172.21.15.55=a;smithi078:172.21.15.78=b;smithi170:172.21.15.170=c;count:3 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:node-exporter ?:9100 0/3 - 64s smithi055=a;smithi078=b;smithi170=c;count:3 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:osd 3 35s ago - 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:prometheus ?:9095 1/1 34s ago 66s smithi078=a;count:1 2024-02-13T14:16:28.211 INFO:teuthology.orchestra.run.smithi055.stdout:rgw.foo.a ?:80 1/1 28s ago 65s smithi055=foo.a;count:1 2024-02-13T14:16:28.529 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:28 smithi055 ceph-mon[25769]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:28.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:28 smithi078 ceph-mon[30086]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:28.878 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-13T14:16:28.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:28 smithi170 ceph-mon[30181]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:29.123 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:29.524 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:29 smithi055 ceph-mon[25769]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:29.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-mon[30086]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:29.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 podman[43088]: 2024-02-13 14:16:29.510162373 +0000 UTC m=+0.121729171 container create 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, vcs-type=git, name=ubi8, build-date=2022-04-29T11:48:11.055897, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vendor=Red Hat, Inc., description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , architecture=x86_64, io.openshift.tags=base rhel8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, version=8.5, io.buildah.version=1.32.0, io.openshift.expose-services=, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.) 2024-02-13T14:16:29.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:29 smithi170 ceph-mon[30181]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:29.992 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 podman[43088]: 2024-02-13 14:16:29.740651457 +0000 UTC m=+0.352218269 container init 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, release=239.1651231664, version=8.5, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, build-date=2022-04-29T11:48:11.055897, vcs-type=git, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, distribution-scope=public, vendor=Red Hat, Inc., architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, name=ubi8, description=Ceph Grafana Container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.expose-services=, com.redhat.component=ubi8-container) 2024-02-13T14:16:29.993 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 podman[43088]: 2024-02-13 14:16:29.746210641 +0000 UTC m=+0.357777441 container start 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, release=239.1651231664, vcs-type=git, version=8.5, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.buildah.version=1.32.0, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., com.redhat.component=ubi8-container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, io.openshift.tags=base rhel8, io.openshift.expose-services=, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, description=Ceph Grafana Container, maintainer=Paul Cuzner , architecture=x86_64, vendor=Red Hat, Inc., build-date=2022-04-29T11:48:11.055897, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-13T14:16:29.993 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 bash[43088]: 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 2024-02-13T14:16:29.993 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 systemd[1]: Started Ceph grafana.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:29.993 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902139248Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-13T14:16:29.993 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902363472Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902388642Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902401327Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902412678Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902423573Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.90243485Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-13T14:16:29.994 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902445309Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902459411Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902470296Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902480141Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902490448Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902501411Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-13T14:16:29.995 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=settings t=2024-02-13T14:16:29.902512649Z level=info msg="App mode production" 2024-02-13T14:16:29.996 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore t=2024-02-13T14:16:29.902573864Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-13T14:16:29.996 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore t=2024-02-13T14:16:29.902598454Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-13T14:16:29.996 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.90370483Z level=info msg="Starting DB migrations" 2024-02-13T14:16:29.996 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.903800798Z level=info msg="Executing migration" id="create migration_log table" 2024-02-13T14:16:29.996 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.904354207Z level=info msg="Executing migration" id="create user table" 2024-02-13T14:16:29.997 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.904832583Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-13T14:16:29.997 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.905427123Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-13T14:16:29.997 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.905898857Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-13T14:16:29.997 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.906677495Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-13T14:16:29.998 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.907222803Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.908247249Z level=info msg="Executing migration" id="create user table v2" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.9087707Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.909304321Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.909819496Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.912409095Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.91527302Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.915846818Z level=info msg="Executing migration" id="Update user table charset" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.916253054Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.916804317Z level=info msg="Executing migration" id="Add missing user data" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.917308839Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.917844566Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.920484683Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.923183578Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-13T14:16:29.999 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.924947623Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.925522588Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.925982528Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.92641393Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.929122394Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.931980544Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.934684181Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.93739194Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.940087429Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.940505807Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.943225152Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.943890486Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.946018056Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.948728226Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-13T14:16:30.000 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.949184654Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.951857645Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.954634319Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.957309018Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.959970758Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.962767638Z level=info msg="Executing migration" id="create star table" 2024-02-13T14:16:30.001 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.963668995Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.964534916Z level=info msg="Executing migration" id="create org table v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.965369274Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.96622297Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.967073098Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.967901584Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.968745775Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.969228153Z level=info msg="Executing migration" id="Update org table charset" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.970057927Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.970939833Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.971758026Z level=info msg="Executing migration" id="create dashboard table" 2024-02-13T14:16:30.002 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.972567289Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.973473182Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.974335677Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.975228055Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.976042611Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.976877798Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.978159829Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.979046224Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.979895772Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.980808698Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.981637008Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.982480388Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.983334232Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.984227085Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.985099417Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-13T14:16:30.003 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.98599595Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.986832326Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.987736527Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.988642877Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.989524094Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.990378256Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.991243612Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.992047191Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.99275249Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-13T14:16:30.004 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.993637689Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-13T14:16:30.242 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.994511751Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-13T14:16:30.242 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:29 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:29.997351919Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-13T14:16:30.242 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.000164779Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-13T14:16:30.242 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.003052202Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.005904177Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.008691146Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.011473187Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.014549505Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.017216733Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.020155423Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.023059124Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.02579492Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.028820394Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.031688522Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.034766855Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.037459238Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.040275882Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-13T14:16:30.243 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.043141374Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.046181031Z level=info msg="Executing migration" id="create data_source table" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.048828314Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.049460566Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.051610748Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.054357956Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.056937583Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.059654898Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.062236196Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.065036149Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.067784489Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.070350292Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.073023318Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.075737798Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-13T14:16:30.245 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.078420846Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.078769812Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.079308883Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.081969443Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.082290008Z level=info msg="Executing migration" id="Add uid column" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.082789029Z level=info msg="Executing migration" id="Update uid value" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.083129485Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.085635455Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.088355671Z level=info msg="Executing migration" id="create api_key table" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.091065214Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.093569732Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.096089867Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.09880832Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.101470351Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.101891349Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-13T14:16:30.246 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.104500549Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.107150394Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.109887618Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.110323483Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.113062865Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.115714791Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.118401639Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-13T14:16:30.247 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.118764548Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.121422147Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.124074853Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.126584401Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.126924309Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.12772377Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.128491096Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.129295579Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.129679439Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.130453549Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-13T14:16:30.248 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.131255607Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.132023424Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.132434158Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.133184264Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.133483958Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.13400834Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.134592543Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.134876931Z level=info msg="Executing migration" id="create quota table v1" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.135279006Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.135683221Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.136439173Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.136833597Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.137261846Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.138028689Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.13833056Z level=info msg="Executing migration" id="create session table" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.139228531Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-13T14:16:30.249 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.140045788Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.140894407Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.141662529Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.142490385Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.143358843Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.144174278Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.144471286Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.14525912Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.145652966Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.146093616Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.146712525Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.146995996Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.147592767Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.148196544Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.150774129Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-13T14:16:30.250 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.15345697Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.156345645Z level=info msg="Executing migration" id="create alert table v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.156843084Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.159450345Z level=info msg="Executing migration" id="add index alert state" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.161988657Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.16476295Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.165160036Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.167745554Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.170521974Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.173146874Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.17354282Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.176249309Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.178849546Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.181497359Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.181896772Z level=info msg="Executing migration" id="Add column is_default" 2024-02-13T14:16:30.251 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.184461541Z level=info msg="Executing migration" id="Add column frequency" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.187048881Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.189517103Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.190152042Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.192881782Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.19543207Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.19811396Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.200602774Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.203245691Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.206037718Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.20868885Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.209135545Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.21172799Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.214355846Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.216987079Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-13T14:16:30.252 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.217455635Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.220041627Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.220714889Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.222734071Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.223225684Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.22366066Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.224007382Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.224493416Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.224932772Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-13T14:16:30.253 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.225696873Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.226509055Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.227214387Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.228021432Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.228796435Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.229606667Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.230031081Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.230810128Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.231657198Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.232468404Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.232898146Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.234490522Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.235299885Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-13T14:16:30.254 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.236055041Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.236393009Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.237260986Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.237608435Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.238326916Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.239100363Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.239901142Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.240732809Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.241474465Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.242435065Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.243059033Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.243480929Z level=info msg="Executing migration" id="Move region to single row" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.244315665Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-13T14:16:30.255 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.244876985Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.245490489Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.246302872Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.246894704Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.247453625Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.250023704Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.25046063Z level=info msg="Executing migration" id="create test_data table" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.252974919Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-13T14:16:30.493 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.2557076Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.256163733Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.258825202Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.261694109Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.264491134Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.264806187Z level=info msg="Executing migration" id="create team table" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.267600475Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.270247939Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.272907675Z level=info msg="Executing migration" id="create team member table" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.273325479Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-13T14:16:30.494 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.276138953Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.278854263Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.281759532Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.282522042Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.283305667Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.284160697Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.284998444Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.285476535Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.28626845Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-13T14:16:30.495 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.286730066Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.287609082Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.288404031Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.288861822Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.289310276Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.289811574Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.290156675Z level=info msg="Executing migration" id="create tag table" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.290568346Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.2910254Z level=info msg="Executing migration" id="create login attempt table" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.291446272Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.291910883Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.292357228Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.294268436Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-13T14:16:30.496 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.294679805Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.295137827Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.295484324Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.295853656Z level=info msg="Executing migration" id="create user auth table" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.296278905Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.296719129Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.297013813Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.297784237Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.298569474Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.299346691Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.300150677Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.300598079Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.301428499Z level=info msg="Executing migration" id="create server_lock table" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.30185054Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.302329014Z level=info msg="Executing migration" id="create user auth token table" 2024-02-13T14:16:30.497 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.302788577Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.303609592Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.304095681Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.304539083Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.305395466Z level=info msg="Executing migration" id="create cache_data table" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.305829844Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.306303936Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.306771779Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.307534288Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.30785281Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.308200764Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.308725821Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.30949764Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.310385845Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-13T14:16:30.498 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.311198361Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.311913452Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.312918474Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.31359061Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.314439973Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.315309055Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.316218655Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.317090418Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.317917212Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.318897494Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.319666394Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.320589716Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.321331498Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.321761207Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.322709257Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-13T14:16:30.499 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.323599132Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.324509665Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.325438934Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.326314617Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.329844318Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.333268601Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.334227291Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.335115151Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.33614108Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-13T14:16:30.500 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.336572799Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.339540981Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.342422739Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.345415776Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.348212929Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.351137056Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.354003691Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.357077197Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.359643469Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-13T14:16:30.501 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.362721622Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.365366249Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.368394207Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.371449746Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.374198958Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.377230363Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.380033688Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.382902705Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.385720159Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.38619924Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.388710484Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.391294306Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.394170023Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.397068653Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.399827033Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-13T14:16:30.502 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.402615937Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.405286798Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.4079841Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.410745205Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.413658594Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.416260634Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.419083801Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.421861475Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.424513868Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.427263535Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.430013887Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.432598944Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.435435489Z level=info msg="Executing migration" id="create alert_image table" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.43616412Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-13T14:16:30.503 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.437183914Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.437913438Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.438705586Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.439165932Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.439251059Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.440010225Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.440983233Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.441747104Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.441890255Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.44201497Z level=info msg="alerts found to migrate" alerts=0 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.442620971Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.443495931Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.444383257Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.445173148Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-13T14:16:30.504 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.446063198Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.446910459Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.447784208Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.448688235Z level=info msg="Executing migration" id="create data_keys table" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.449439943Z level=info msg="Executing migration" id="create secrets table" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.450398188Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.454772315Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.455706503Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.456593907Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.460858599Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.465052644Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.468024442Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.470705829Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.47346867Z level=info msg="Executing migration" id="create permission table" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.4762078Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.479138018Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-13T14:16:30.505 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.481712559Z level=info msg="Executing migration" id="create role table" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.484368295Z level=info msg="Executing migration" id="add column display_name" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.486866293Z level=info msg="Executing migration" id="add column group_name" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.489231034Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.491778191Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.494270828Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-13T14:16:30.506 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.496920364Z level=info msg="Executing migration" id="create team role table" 2024-02-13T14:16:30.747 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.499382066Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-13T14:16:30.747 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.502084947Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-13T14:16:30.747 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.504399453Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.50696878Z level=info msg="Executing migration" id="create user role table" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.509698332Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.512263002Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.515099597Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.517617839Z level=info msg="Executing migration" id="create builtin role table" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.52024065Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.522702919Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.525383195Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.528057662Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.530699829Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.531330157Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.532105948Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.532826325Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-13T14:16:30.748 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.53365854Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.534579926Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.53570683Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.536485002Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.537284697Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.538055815Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.538975082Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.53964094Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.540493645Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.540904575Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.541265847Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.542047723Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.543005167Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.544156389Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.545046249Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-13T14:16:30.749 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.545832327Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.546511588Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.547362535Z level=info msg="Executing migration" id="add correlation config column" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.548392988Z level=info msg="Executing migration" id="create entity_events table" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.549178342Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.550134114Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.55026006Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:30.750 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.550822838Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.550911357Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.551625021Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.552504306Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.553307367Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.554120428Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.555041586Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.555765878Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.556701614Z level=info msg="Executing migration" id="Drop public config table" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.557431812Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.558263759Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.559230002Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.56176712Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.564383993Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-13T14:16:30.751 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.567707994Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.570657426Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.573424994Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.575999352Z level=info msg="Executing migration" id="add share column" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.578985602Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.581439113Z level=info msg="Executing migration" id="create file table" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.58415145Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.586850238Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.589639466Z level=info msg="Executing migration" id="create file_meta table" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.592475706Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.595249351Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.597888892Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.600853561Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.603637712Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-13T14:16:30.752 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.606110963Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.609012192Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.61160953Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.614081112Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.616798387Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.619510732Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.622048916Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.624461423Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.626909962Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.629347462Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.639059615Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.641736516Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.644462449Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.647823222Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.650382676Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-13T14:16:30.753 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.652869882Z level=info msg="Executing migration" id="create folder table" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.655582543Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.658048919Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.66074727Z level=info msg="Executing migration" id="Update folder title length" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.661418468Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=migrator t=2024-02-13T14:16:30.662271711Z level=info msg="migrations completed" performed=485 skipped=0 duration=758.499059ms 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore t=2024-02-13T14:16:30.662863374Z level=info msg="Created default organization" 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=plugin.loader t=2024-02-13T14:16:30.717775734Z level=info msg="Plugin registered" pluginID=input 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=plugin.loader t=2024-02-13T14:16:30.746223561Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=plugin.loader t=2024-02-13T14:16:30.746257049Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-13T14:16:30.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=secrets t=2024-02-13T14:16:30.74630922Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-13T14:16:30.945 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:30.946 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:30.946 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:30.946 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:30.946 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:30.946 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:30 smithi055 ceph-mon[25769]: Deploying daemon node-exporter.a on smithi055 2024-02-13T14:16:31.019 INFO:teuthology.orchestra.run.smithi055.stdout:HOST ADDR LABELS STATUS 2024-02-13T14:16:31.020 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 172.21.15.55 2024-02-13T14:16:31.020 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 172.21.15.78 2024-02-13T14:16:31.020 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 172.21.15.170 2024-02-13T14:16:31.020 INFO:teuthology.orchestra.run.smithi055.stdout:3 hosts in cluster 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.156 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:30 smithi170 ceph-mon[30181]: Deploying daemon node-exporter.a on smithi055 2024-02-13T14:16:31.198 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=query_data t=2024-02-13T14:16:30.748532686Z level=info msg="Query Service initialization" 2024-02-13T14:16:31.198 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=live.push_http t=2024-02-13T14:16:30.750997432Z level=info msg="Live Push Gateway initialization" 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:31.198 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:30 smithi078 ceph-mon[30086]: Deploying daemon node-exporter.a on smithi055 2024-02-13T14:16:31.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=infra.usagestats.collector t=2024-02-13T14:16:31.353600256Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-13T14:16:31.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=provisioning.datasources t=2024-02-13T14:16:31.354540403Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-13T14:16:31.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=provisioning.datasources t=2024-02-13T14:16:31.365310413Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-13T14:16:31.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=provisioning.alerting t=2024-02-13T14:16:31.376119734Z level=info msg="starting to provision alerting" 2024-02-13T14:16:31.698 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=provisioning.alerting t=2024-02-13T14:16:31.376138571Z level=info msg="finished to provision alerting" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=grafanaStorageLogger t=2024-02-13T14:16:31.376263287Z level=info msg="storage starting" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=ngalert.state.manager t=2024-02-13T14:16:31.376610014Z level=info msg="Warming state cache for startup" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=http.server t=2024-02-13T14:16:31.378155598Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=ngalert.state.manager t=2024-02-13T14:16:31.385136154Z level=info msg="State cache has been initialized" states=0 duration=8.525254ms 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=ticker t=2024-02-13T14:16:31.385191112Z level=info msg=starting first_tick=2024-02-13T14:16:40Z 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=ngalert.multiorg.alertmanager t=2024-02-13T14:16:31.385195992Z level=info msg="starting MultiOrg Alertmanager" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore.transactions t=2024-02-13T14:16:31.391747435Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore.transactions t=2024-02-13T14:16:31.396920243Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-13T14:16:31.699 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:16:31 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a[43111]: logger=sqlstore.transactions t=2024-02-13T14:16:31.405453512Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-13T14:16:31.723 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-13T14:16:31.984 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:32.064 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:31 smithi055 bash[58997]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-13T14:16:32.356 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:32 smithi055 ceph-mon[25769]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:32.356 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:32 smithi055 ceph-mon[25769]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:32.356 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:32 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:32.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:32 smithi170 ceph-mon[30181]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:32.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:32 smithi170 ceph-mon[30181]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:32.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:32 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:32.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:32 smithi078 ceph-mon[30086]: from='client.14550 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:32.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:32 smithi078 ceph-mon[30086]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:32.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:32 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:32.814 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:32 smithi055 bash[58997]: Getting image source signatures 2024-02-13T14:16:32.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:32 smithi055 bash[58997]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-13T14:16:32.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:32 smithi055 bash[58997]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-13T14:16:32.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:32 smithi055 bash[58997]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-13T14:16:33.303 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:33 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:16:33] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:34.243 INFO:teuthology.orchestra.run.smithi055.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736102VS4P0KGN 3726G No 2m ago Has a FileSystem, LVM detected 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme1n1 ssd Linux_1703b7d952667888231c 89.4G Yes 2m ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme2n1 ssd Linux_0b05971227b046f88a4f 89.4G Yes 2m ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme3n1 ssd Linux_0ab65186b915a0c8cc2a 89.4G Yes 2m ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi055 /dev/nvme4n1 ssd Linux_97dd25e9fc01671a7000 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400QR400BGN 372G No 106s ago Has a FileSystem, LVM detected 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 /dev/nvme1n1 ssd Linux_338a0b4a8d3b14a42642 89.4G Yes 106s ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 /dev/nvme2n1 ssd Linux_01255163cc4ddf503269 89.4G Yes 106s ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 /dev/nvme3n1 ssd Linux_919fc64e60babb55c756 89.4G Yes 106s ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi078 /dev/nvme4n1 ssd Linux_95f547568fb8f4a65f38 89.4G No 106s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002R480DGN 447G No 82s ago Has a FileSystem, LVM detected 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 /dev/nvme1n1 ssd Linux_a6dd76a1bacd52ba1f6b 89.4G Yes 82s ago 2024-02-13T14:16:34.244 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 /dev/nvme2n1 ssd Linux_83df34e284b725f9b2c6 89.4G Yes 82s ago 2024-02-13T14:16:34.245 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 /dev/nvme3n1 ssd Linux_a77f88c7e758642c7773 89.4G Yes 82s ago 2024-02-13T14:16:34.245 INFO:teuthology.orchestra.run.smithi055.stdout:smithi170 /dev/nvme4n1 ssd Linux_55782199e3e9a91978aa 89.4G No 82s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-13T14:16:34.314 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:34 smithi055 ceph-mon[25769]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:34.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:34 smithi170 ceph-mon[30181]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:34.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:34 smithi078 ceph-mon[30086]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:35.258 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-13T14:16:35.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-mon[25769]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:35.565 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 bash[58997]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-13T14:16:35.565 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 bash[58997]: Writing manifest to image destination 2024-02-13T14:16:35.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:35 smithi170 ceph-mon[30181]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:35.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:35 smithi078 ceph-mon[30086]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:35.972 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 podman[58997]: 2024-02-13 14:16:35.654568756 +0000 UTC m=+3.868298987 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-13T14:16:35.972 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 podman[58997]: 2024-02-13 14:16:35.744940234 +0000 UTC m=+3.958670506 container create d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:35.973 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 podman[58997]: 2024-02-13 14:16:35.971971438 +0000 UTC m=+4.185701669 container init d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:36.069 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 podman[58997]: 2024-02-13 14:16:35.977656811 +0000 UTC m=+4.191387042 container start d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.981Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.981Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T14:16:36.233 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=arp 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=edac 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-13T14:16:36.234 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=os 2024-02-13T14:16:36.235 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=stat 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-13T14:16:36.236 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=time 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=uname 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.982Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.983Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:35 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a[59292]: ts=2024-02-13T14:16:35.983Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:36 smithi055 bash[58997]: d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 2024-02-13T14:16:36.237 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:16:36 smithi055 systemd[1]: Started Ceph node-exporter.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:36.501 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:36 smithi055 ceph-mon[25769]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:36.501 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:36 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.502 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:36 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.502 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:36 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:36 smithi170 ceph-mon[30181]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:36.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:36 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:36 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:36 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:36 smithi078 ceph-mon[30086]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:36.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:36 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:36 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:36.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:36 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' 2024-02-13T14:16:37.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:37 smithi078 ceph-mon[30086]: Deploying daemon node-exporter.b on smithi078 2024-02-13T14:16:37.449 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:37 smithi078 ceph-mon[30086]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:37.449 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:37 smithi078 bash[43380]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-13T14:16:37.541 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:37 smithi055 ceph-mon[25769]: Deploying daemon node-exporter.b on smithi078 2024-02-13T14:16:37.541 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:37 smithi055 ceph-mon[25769]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:37 smithi170 ceph-mon[30181]: Deploying daemon node-exporter.b on smithi078 2024-02-13T14:16:37.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:37 smithi170 ceph-mon[30181]: from='mgr.14436 172.21.15.55:0/3260163254' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-13T14:16:38.120 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: alertmanager 2024-02-13T14:16:38.120 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: alertmanager 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: count: 1 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi055=a 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:15:26.951720Z' 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: ports: 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: - 9093 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: - 9094 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: running: 0 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout: size: 1 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:events: 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:- 2024-02-13T14:16:14.208759Z service:alertmanager [INFO] "service was created" 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.121 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: grafana 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: grafana 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: count: 1 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi078=a 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout:spec: 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: anonymous_access: true 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: protocol: https 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:15:29.612923Z' 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: ports: 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: - 3000 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: running: 0 2024-02-13T14:16:38.122 INFO:teuthology.orchestra.run.smithi055.stdout: size: 1 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:events: 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:- 2024-02-13T14:16:29.920028Z service:grafana [INFO] "service was created" 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: mgr 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: mgr 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: count: 3 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi055=y 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi078=x 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi170=z 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:13:51.861092Z' 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: last_refresh: '2024-02-13T14:15:52.814005Z' 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: running: 3 2024-02-13T14:16:38.123 INFO:teuthology.orchestra.run.smithi055.stdout: size: 3 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: mon 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: mon 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: count: 3 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi055:172.21.15.55=a 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi078:172.21.15.78=b 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi170:172.21.15.170=c 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:13:26.119930Z' 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: last_refresh: '2024-02-13T14:15:52.813873Z' 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: running: 3 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout: size: 3 2024-02-13T14:16:38.124 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: node-exporter 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: node-exporter 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: count: 3 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi055=a 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi078=b 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi170=c 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:15:24.203489Z' 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: ports: 2024-02-13T14:16:38.125 INFO:teuthology.orchestra.run.smithi055.stdout: - 9100 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: running: 0 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: size: 3 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:events: 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:- 2024-02-13T14:16:36.149573Z service:node-exporter [INFO] "service was created" 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: osd 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: osd 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:unmanaged: true 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:spec: 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: filter_logic: AND 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: objectstore: bluestore 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: container_image_id: 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:16:38.126 INFO:teuthology.orchestra.run.smithi055.stdout: last_refresh: '2024-02-13T14:15:52.814102Z' 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: running: 3 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: size: 3 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: prometheus 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: prometheus 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: count: 1 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi078=a 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:15:21.600247Z' 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: last_refresh: '2024-02-13T14:15:53.567589Z' 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: ports: 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: - 9095 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: running: 1 2024-02-13T14:16:38.127 INFO:teuthology.orchestra.run.smithi055.stdout: size: 1 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:--- 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:service_type: rgw 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:service_id: foo.a 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:service_name: rgw.foo.a 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:placement: 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: count: 1 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: hosts: 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: - smithi055=foo.a 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout:status: 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: created: '2024-02-13T14:15:22.541536Z' 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: last_refresh: '2024-02-13T14:15:59.928700Z' 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: ports: 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: - 80 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: running: 1 2024-02-13T14:16:38.128 INFO:teuthology.orchestra.run.smithi055.stdout: size: 1 2024-02-13T14:16:38.198 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:37 smithi078 bash[43380]: Getting image source signatures 2024-02-13T14:16:38.198 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:37 smithi078 bash[43380]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-13T14:16:38.198 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:37 smithi078 bash[43380]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-13T14:16:38.198 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:37 smithi078 bash[43380]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-13T14:16:38.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:38 smithi055 ceph-mon[25769]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:38.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:38 smithi170 ceph-mon[30181]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:38.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:38 smithi078 ceph-mon[30086]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:38.979 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 9e367004-ca79-11ee-95b9-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-13T14:16:39.226 INFO:teuthology.orchestra.run.smithi055.stderr:Inferring config /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/mon.a/config 2024-02-13T14:16:39.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:39 smithi055 ceph-mon[25769]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-13T14:16:39.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:39 smithi170 ceph-mon[30181]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-13T14:16:39.698 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:39 smithi078 ceph-mon[30086]: from='client.14562 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-13T14:16:40.443 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:40 smithi055 ceph-mon[25769]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:40.656 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:40 smithi170 ceph-mon[30181]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:40.697 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:40 smithi078 ceph-mon[30086]: pgmap v27: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:41.351 INFO:teuthology.orchestra.run.smithi055.stdout:osd 3 48s ago - 2024-02-13T14:16:42.027 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-13T14:16:42.036 INFO:tasks.cephadm:Teardown begin 2024-02-13T14:16:42.036 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:42.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:42.104 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:16:42.138 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-13T14:16:42.139 DEBUG:teuthology.orchestra.run.smithi055:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:16:42.153 DEBUG:teuthology.orchestra.run.smithi078:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:16:42.171 DEBUG:teuthology.orchestra.run.smithi170:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-13T14:16:42.195 INFO:tasks.cephadm:Stopping all daemons... 2024-02-13T14:16:42.196 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-13T14:16:42.196 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a 2024-02-13T14:16:42.303 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 systemd[1]: Stopping Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:42.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:42 smithi170 ceph-mon[30181]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:42.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:42 smithi170 ceph-mon[30181]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:42.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:42 smithi078 ceph-mon[30086]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:42.448 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:42 smithi078 ceph-mon[30086]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE Bus STOPPING 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE Bus STOPPED 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE Bus STARTING 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-mon[25769]: pgmap v28: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-mon[25769]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-13T14:16:42.565 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a[25746]: 2024-02-13T14:16:42.497+0000 7f9dbb7b5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:42.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a[25746]: 2024-02-13T14:16:42.497+0000 7f9dbb7b5700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-13T14:16:42.566 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 podman[59774]: 2024-02-13 14:16:42.548723996 +0000 UTC m=+0.168182119 container stop 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-02-13T14:16:42.854 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 podman[59774]: 2024-02-13 14:16:42.574380997 +0000 UTC m=+0.193839163 container died 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-13T14:16:42.855 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE Serving on http://:::9283 2024-02-13T14:16:42.855 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:42 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:42] ENGINE Bus STARTED 2024-02-13T14:16:43.163 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:43 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: ::ffff:172.21.15.78 - - [13/Feb/2024:14:16:43] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-13T14:16:43.163 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 podman[59774]: 2024-02-13 14:16:42.969250459 +0000 UTC m=+0.588708580 container cleanup 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-13T14:16:43.163 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:42 smithi055 bash[59774]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a 2024-02-13T14:16:43.401 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service' 2024-02-13T14:16:43.425 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:43 smithi055 podman[59804]: 2024-02-13 14:16:43.162190787 +0000 UTC m=+0.608431232 container remove 673ad2dec54134fd96063eba7ca93ac9ce410365e458e798e44c074ddb1013b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-a, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-02-13T14:16:43.425 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:43 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-13T14:16:43.425 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:43 smithi055 systemd[1]: Stopped Ceph mon.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:43.426 INFO:journalctl@ceph.mon.a.smithi055.stdout:Feb 13 14:16:43 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.a.service: Consumed 7.625s CPU time. 2024-02-13T14:16:43.447 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:43.448 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-13T14:16:43.448 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-13T14:16:43.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.b 2024-02-13T14:16:43.948 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:43 smithi078 systemd[1]: Stopping Ceph mon.b for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:44.572 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:44 smithi078 bash[43380]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-13T14:16:44.572 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:44 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b[30063]: 2024-02-13T14:16:44.382+0000 7f66aeaa5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:44.572 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:44 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b[30063]: 2024-02-13T14:16:44.382+0000 7f66aeaa5700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-13T14:16:44.572 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:44 smithi078 podman[43511]: 2024-02-13 14:16:44.403924931 +0000 UTC m=+0.192399323 container died 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE Bus STOPPING 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE Bus STOPPED 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE Bus STARTING 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE Serving on http://:::9283 2024-02-13T14:16:44.815 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:44 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: [13/Feb/2024:14:16:44] ENGINE Bus STARTED 2024-02-13T14:16:44.948 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:44 smithi078 bash[43380]: Writing manifest to image destination 2024-02-13T14:16:45.261 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43380]: 2024-02-13 14:16:45.00209253 +0000 UTC m=+7.803444185 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-13T14:16:45.261 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43511]: 2024-02-13 14:16:45.109160103 +0000 UTC m=+0.897634495 container cleanup 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:16:45.261 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:45 smithi078 bash[43511]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b 2024-02-13T14:16:45.515 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43380]: 2024-02-13 14:16:45.42737643 +0000 UTC m=+8.228728071 container create e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:16:45.515 INFO:journalctl@ceph.mon.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43530]: 2024-02-13 14:16:45.26022437 +0000 UTC m=+0.857675507 container remove 37fae4a395b58a8d3877949da3b733da0ac4a692b5402cf8b6f8fca16f12229a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, CEPH_POINT_RELEASE=) 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43380]: 2024-02-13 14:16:45.637717961 +0000 UTC m=+8.439069607 container init e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 podman[43380]: 2024-02-13 14:16:45.643400341 +0000 UTC m=+8.444751982 container start e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.646Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.646Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-13T14:16:45.773 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=arp 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-13T14:16:45.774 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=edac 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-13T14:16:45.775 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=os 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.647Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-13T14:16:45.776 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=stat 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=time 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=uname 2024-02-13T14:16:45.777 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b[43576]: ts=2024-02-13T14:16:45.648Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 bash[43380]: e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e 2024-02-13T14:16:45.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:16:45 smithi078 systemd[1]: Started Ceph node-exporter.b for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:16:45.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.b.service' 2024-02-13T14:16:45.840 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:45.840 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-13T14:16:45.840 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-13T14:16:45.840 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.c 2024-02-13T14:16:46.147 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:45 smithi170 systemd[1]: Stopping Ceph mon.c for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:46.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:46 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c[30158]: 2024-02-13T14:16:46.144+0000 7fa3b4927700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:46.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:46 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c[30158]: 2024-02-13T14:16:46.144+0000 7fa3b4927700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-13T14:16:46.406 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:46 smithi170 podman[43391]: 2024-02-13 14:16:46.278880369 +0000 UTC m=+0.240970130 container died a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, ceph=True) 2024-02-13T14:16:46.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:46 smithi170 podman[43391]: 2024-02-13 14:16:46.75902222 +0000 UTC m=+0.721112022 container cleanup a50dafabb547d93db097065bbfbf190d193db2fc41efe05ebe1bc234c8b2ec86 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2) 2024-02-13T14:16:46.906 INFO:journalctl@ceph.mon.c.smithi170.stdout:Feb 13 14:16:46 smithi170 bash[43391]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mon-c 2024-02-13T14:16:47.190 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mon.c.service' 2024-02-13T14:16:47.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:47.235 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-13T14:16:47.235 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-13T14:16:47.236 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y 2024-02-13T14:16:47.565 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:47 smithi055 systemd[1]: Stopping Ceph mgr.y for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:47.875 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:16:47.575+0000 7fed6b77c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:47.875 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:47 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y[26024]: 2024-02-13T14:16:47.575+0000 7fed6b77c700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-13T14:16:47.875 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:47 smithi055 podman[59928]: 2024-02-13 14:16:47.607421816 +0000 UTC m=+0.178526474 container died 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2) 2024-02-13T14:16:48.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:48 smithi055 podman[59928]: 2024-02-13 14:16:48.073548952 +0000 UTC m=+0.644653608 container cleanup 754d50c91a1dc26950763fce468c9d684ea320b5028e12786f41a4f4cc1a442e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:e1b79d5a4193a484176863ab96c0ca28c00e1662, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-02-13T14:16:48.315 INFO:journalctl@ceph.mgr.y.smithi055.stdout:Feb 13 14:16:48 smithi055 bash[59928]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-y 2024-02-13T14:16:48.507 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.y.service' 2024-02-13T14:16:48.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:48.543 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-13T14:16:48.543 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-13T14:16:48.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.x 2024-02-13T14:16:48.856 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:16:48 smithi078 systemd[1]: Stopping Ceph mgr.x for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:49.159 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:16:48 smithi078 podman[43676]: 2024-02-13 14:16:48.855654014 +0000 UTC m=+0.121073682 container died cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-13T14:16:49.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:16:49 smithi078 podman[43676]: 2024-02-13 14:16:49.248273714 +0000 UTC m=+0.513693384 container cleanup cab1c8d4cff1b27f6e15b4a893e5f9cad8ffcee011686813171e7a5fa5a0f9a9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-13T14:16:49.448 INFO:journalctl@ceph.mgr.x.smithi078.stdout:Feb 13 14:16:49 smithi078 bash[43676]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-x 2024-02-13T14:16:49.591 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.x.service' 2024-02-13T14:16:49.628 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:49.629 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-13T14:16:49.629 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-13T14:16:49.629 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.z 2024-02-13T14:16:50.012 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:16:49 smithi170 systemd[1]: Stopping Ceph mgr.z for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:50.306 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:16:50 smithi170 podman[43524]: 2024-02-13 14:16:50.011727974 +0000 UTC m=+0.189278561 container died a56913adfd5103788942e0f9c5618aac491d15495bf367c38a911b71921b97d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-02-13T14:16:50.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:16:50 smithi170 podman[43524]: 2024-02-13 14:16:50.387737447 +0000 UTC m=+0.565288033 container cleanup a56913adfd5103788942e0f9c5618aac491d15495bf367c38a911b71921b97d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-13T14:16:50.656 INFO:journalctl@ceph.mgr.z.smithi170.stdout:Feb 13 14:16:50 smithi170 bash[43524]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-mgr-z 2024-02-13T14:16:50.793 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@mgr.z.service' 2024-02-13T14:16:50.830 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:50.830 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-13T14:16:50.831 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-13T14:16:50.831 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.0 2024-02-13T14:16:51.214 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:50 smithi055 systemd[1]: Stopping Ceph osd.0 for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:51.565 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0[44231]: 2024-02-13T14:16:51.212+0000 7f901fa48700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:51.565 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0[44231]: 2024-02-13T14:16:51.212+0000 7f901fa48700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-13T14:16:51.565 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:51 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0[44231]: 2024-02-13T14:16:51.212+0000 7f901fa48700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:16:56.815 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:56 smithi055 podman[60062]: 2024-02-13 14:16:56.559706903 +0000 UTC m=+5.533349126 container died 7eea229474c3244d6dbbc8fedf94ddd38aa4cafd405d07465eeae6c316352559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-02-13T14:16:57.295 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60062]: 2024-02-13 14:16:57.035657543 +0000 UTC m=+6.009299771 container cleanup 7eea229474c3244d6dbbc8fedf94ddd38aa4cafd405d07465eeae6c316352559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-13T14:16:57.296 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 bash[60062]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0 2024-02-13T14:16:57.564 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60083]: 2024-02-13 14:16:57.295106692 +0000 UTC m=+0.735025757 container remove 7eea229474c3244d6dbbc8fedf94ddd38aa4cafd405d07465eeae6c316352559 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:16:57.982 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60183]: 2024-02-13 14:16:57.562347662 +0000 UTC m=+0.018825646 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:16:57.983 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60183]: 2024-02-13 14:16:57.662652253 +0000 UTC m=+0.119130236 container create 11b3818a5d002834f1b040fb59f79a837df1663c0a0c6cb986def8cd31b2dc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:16:57.983 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60183]: 2024-02-13 14:16:57.881645462 +0000 UTC m=+0.338123450 container init 11b3818a5d002834f1b040fb59f79a837df1663c0a0c6cb986def8cd31b2dc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, GIT_CLEAN=True) 2024-02-13T14:16:57.983 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60183]: 2024-02-13 14:16:57.887283989 +0000 UTC m=+0.343761975 container start 11b3818a5d002834f1b040fb59f79a837df1663c0a0c6cb986def8cd31b2dc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_CLEAN=True) 2024-02-13T14:16:57.983 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:57 smithi055 podman[60183]: 2024-02-13 14:16:57.981563095 +0000 UTC m=+0.438041078 container attach 11b3818a5d002834f1b040fb59f79a837df1663c0a0c6cb986def8cd31b2dc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-02-13T14:16:58.315 INFO:journalctl@ceph.osd.0.smithi055.stdout:Feb 13 14:16:58 smithi055 podman[60183]: 2024-02-13 14:16:58.184189731 +0000 UTC m=+0.640667731 container died 11b3818a5d002834f1b040fb59f79a837df1663c0a0c6cb986def8cd31b2dc22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:16:58.894 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.0.service' 2024-02-13T14:16:58.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:16:58.941 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-13T14:16:58.942 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-13T14:16:58.942 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.1 2024-02-13T14:16:59.239 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:16:58 smithi078 systemd[1]: Stopping Ceph osd.1 for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:16:59.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:16:59 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1[34638]: 2024-02-13T14:16:59.237+0000 7fe0fc993700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:16:59.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:16:59 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1[34638]: 2024-02-13T14:16:59.237+0000 7fe0fc993700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-13T14:16:59.698 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:16:59 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1[34638]: 2024-02-13T14:16:59.237+0000 7fe0fc993700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:17:04.797 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:04 smithi078 podman[43812]: 2024-02-13 14:17:04.539954148 +0000 UTC m=+5.410573975 container died a2d6fbd6047c165062052e3952b04b67360c2a876149d41482198e557517d0af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-13T14:17:05.176 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:04 smithi078 podman[43812]: 2024-02-13 14:17:04.894557312 +0000 UTC m=+5.765177135 container cleanup a2d6fbd6047c165062052e3952b04b67360c2a876149d41482198e557517d0af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-02-13T14:17:05.176 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:04 smithi078 bash[43812]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1 2024-02-13T14:17:05.448 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43846]: 2024-02-13 14:17:05.174828237 +0000 UTC m=+0.637035444 container remove a2d6fbd6047c165062052e3952b04b67360c2a876149d41482198e557517d0af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:17:05.782 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.407325726 +0000 UTC m=+0.018700568 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:17:05.783 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.513070303 +0000 UTC m=+0.124445135 container create feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:17:05.783 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.698837304 +0000 UTC m=+0.310212140 container init feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-13T14:17:05.783 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.70487199 +0000 UTC m=+0.316246824 container start feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-13T14:17:05.783 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.781946512 +0000 UTC m=+0.393321356 container attach feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:17:06.198 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:05 smithi078 podman[43942]: 2024-02-13 14:17:05.990091365 +0000 UTC m=+0.601466202 container died feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:17:06.538 INFO:journalctl@ceph.osd.1.smithi078.stdout:Feb 13 14:17:06 smithi078 podman[43989]: 2024-02-13 14:17:06.534874462 +0000 UTC m=+0.535605666 container remove feab1bfc77060a69ec12c879b0e8ce2ea951b6899dc78855b91a73bf2a4f960b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-1-deactivate, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux ) 2024-02-13T14:17:06.562 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.1.service' 2024-02-13T14:17:06.604 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:17:06.604 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-13T14:17:06.604 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-13T14:17:06.604 DEBUG:teuthology.orchestra.run.smithi170:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.2 2024-02-13T14:17:06.906 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:06 smithi170 systemd[1]: Stopping Ceph osd.2 for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:07.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:06 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2[35448]: 2024-02-13T14:17:06.958+0000 7f8fd20fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:17:07.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:06 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2[35448]: 2024-02-13T14:17:06.958+0000 7f8fd20fd700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-13T14:17:07.406 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:06 smithi170 ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2[35448]: 2024-02-13T14:17:06.958+0000 7f8fd20fd700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-13T14:17:12.846 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:12 smithi170 podman[43656]: 2024-02-13 14:17:12.508024425 +0000 UTC m=+5.708716894 container died d49a9f147f6d9651258140f52c0a9fb4ff4cae2f6d9f04052db800a9009cbe38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-13T14:17:12.846 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:12 smithi170 podman[43656]: 2024-02-13 14:17:12.844489436 +0000 UTC m=+6.045181945 container cleanup d49a9f147f6d9651258140f52c0a9fb4ff4cae2f6d9f04052db800a9009cbe38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:17:13.157 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:12 smithi170 bash[43656]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2 2024-02-13T14:17:13.157 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43679]: 2024-02-13 14:17:13.023128922 +0000 UTC m=+0.512087754 container remove d49a9f147f6d9651258140f52c0a9fb4ff4cae2f6d9f04052db800a9009cbe38 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True) 2024-02-13T14:17:13.777 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.358347373 +0000 UTC m=+0.018491693 image pull 28480620e5edd298d55c0e6673dcbad9ed23edd37269028a11063c1b8925676a quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f 2024-02-13T14:17:13.777 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.465366795 +0000 UTC m=+0.125511109 container create a3fb67dabcee8d7099ac469fb033efeb9a4fd0e23b7004ca31bd50dcb812589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-13T14:17:13.777 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.693177833 +0000 UTC m=+0.353322155 container init a3fb67dabcee8d7099ac469fb033efeb9a4fd0e23b7004ca31bd50dcb812589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-13T14:17:13.777 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.698967806 +0000 UTC m=+0.359112120 container start a3fb67dabcee8d7099ac469fb033efeb9a4fd0e23b7004ca31bd50dcb812589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:17:13.777 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.77598755 +0000 UTC m=+0.436131884 container attach a3fb67dabcee8d7099ac469fb033efeb9a4fd0e23b7004ca31bd50dcb812589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-13T14:17:14.156 INFO:journalctl@ceph.osd.2.smithi170.stdout:Feb 13 14:17:13 smithi170 podman[43773]: 2024-02-13 14:17:13.995412567 +0000 UTC m=+0.655556872 container died a3fb67dabcee8d7099ac469fb033efeb9a4fd0e23b7004ca31bd50dcb812589e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-13T14:17:14.583 DEBUG:teuthology.orchestra.run.smithi170:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@osd.2.service' 2024-02-13T14:17:14.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:17:14.623 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-13T14:17:14.623 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-13T14:17:14.624 DEBUG:teuthology.orchestra.run.smithi055:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a 2024-02-13T14:17:14.934 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:14 smithi055 systemd[1]: Stopping Ceph rgw.foo.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:15.315 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a[50265]: 2024-02-13T14:17:14.932+0000 7f4a4705d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-13T14:17:15.315 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:14 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a[50265]: 2024-02-13T14:17:14.932+0000 7f4a558ceac0 -1 shutting down 2024-02-13T14:17:25.248 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:24 smithi055 bash[60314]: time="2024-02-13T14:17:24Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-13T14:17:25.248 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:24 smithi055 podman[60314]: 2024-02-13 14:17:24.955480438 +0000 UTC m=+10.142094942 container died b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5) 2024-02-13T14:17:25.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 podman[60314]: 2024-02-13 14:17:25.438105555 +0000 UTC m=+10.624720101 container cleanup b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, ceph=True, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.2) 2024-02-13T14:17:25.565 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 bash[60314]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a 2024-02-13T14:17:25.826 DEBUG:teuthology.orchestra.run.smithi055:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service' 2024-02-13T14:17:25.850 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 podman[60351]: 2024-02-13 14:17:25.681013999 +0000 UTC m=+0.719596069 container remove b9a0e6009978dd664e4e45a3d970fe35d95d4a0754ca42523d0f191a3d03f1a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:48116d68f53d3a2ae4d119fcfc94a11cef23cee69b8a381eaa5186380a06fd9f, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-rgw-foo-a, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-phlogistonjohn-testing-2024-02-12-1654-e1b79d5, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-13T14:17:25.851 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service: Main process exited, code=exited, status=137/n/a 2024-02-13T14:17:25.851 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service: Failed with result 'exit-code'. 2024-02-13T14:17:25.851 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 systemd[1]: Stopped Ceph rgw.foo.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:25.851 INFO:journalctl@ceph.rgw.foo.a.smithi055.stdout:Feb 13 14:17:25 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@rgw.foo.a.service: Consumed 1.406s CPU time. 2024-02-13T14:17:25.863 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:17:25.864 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-13T14:17:25.864 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-13T14:17:25.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ceph-9e367004-ca79-11ee-95b9-87774f69a715@prometheus.a 2024-02-13T14:17:26.198 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:25 smithi078 systemd[1]: Stopping Ceph prometheus.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:26.512 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-13T14:17:26.513 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a[41234]: ts=2024-02-13T14:17:26.238Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-13T14:17:26.514 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 podman[44074]: 2024-02-13 14:17:26.24851771 +0000 UTC m=+0.197008501 container died 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:26.861 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 podman[44074]: 2024-02-13 14:17:26.601714768 +0000 UTC m=+0.550205567 container cleanup 91836debb8c6289ca53efff2a926438077358a8c832fa32bf15603ebdae915a1 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:26.861 INFO:journalctl@ceph.prometheus.a.smithi078.stdout:Feb 13 14:17:26 smithi078 bash[44074]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-prometheus-a 2024-02-13T14:17:27.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo pkill -f 'journalctl -f -n 0 -u ceph-9e367004-ca79-11ee-95b9-87774f69a715@prometheus.a.service' 2024-02-13T14:17:27.062 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-13T14:17:27.062 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-13T14:17:27.062 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T14:17:27.257 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:28.315 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:28 smithi055 systemd[1]: Stopping Ceph alertmanager.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:28.652 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:28 smithi055 ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a[57460]: ts=2024-02-13T14:17:28.336Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-13T14:17:28.653 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:28 smithi055 podman[60670]: 2024-02-13 14:17:28.358608797 +0000 UTC m=+0.125714934 container died cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:29.065 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:28 smithi055 podman[60670]: 2024-02-13 14:17:28.842843616 +0000 UTC m=+0.609949795 container cleanup cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:29.065 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:28 smithi055 bash[60670]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a 2024-02-13T14:17:29.518 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:29 smithi055 podman[60689]: 2024-02-13 14:17:29.111152655 +0000 UTC m=+0.754167641 container remove cbd7f0df475e3d845259e686fc14ffbb5d3fbcc6945b29e36e5348459ae29bff (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:29.519 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:29 smithi055 podman[60689]: 2024-02-13 14:17:29.186731552 +0000 UTC m=+0.829746489 volume remove 66d5df4e69b6c4ddc5ecc221f27cbbcf49cbb22d6044fc151ae014ff35d9c69d 2024-02-13T14:17:29.519 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:29 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-13T14:17:29.519 INFO:journalctl@ceph.alertmanager.a.smithi055.stdout:Feb 13 14:17:29 smithi055 systemd[1]: Stopped Ceph alertmanager.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:29.779 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:29 smithi055 systemd[1]: Stopping Ceph node-exporter.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:30.065 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:29 smithi055 podman[60808]: 2024-02-13 14:17:29.778495 +0000 UTC m=+0.122464185 container died d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:30.510 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 podman[60808]: 2024-02-13 14:17:30.33089166 +0000 UTC m=+0.674860837 container cleanup d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:30.510 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 bash[60808]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a 2024-02-13T14:17:30.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 podman[60826]: 2024-02-13 14:17:30.632408519 +0000 UTC m=+0.859300892 container remove d28354d3cfbfad41140c1305292e9ff6fc601ed1e0c3f32f3b931cd6822a5da9 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-13T14:17:30.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:17:30.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-13T14:17:30.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 systemd[1]: Stopped Ceph node-exporter.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:30.815 INFO:journalctl@ceph.node-exporter.a.smithi055.stdout:Feb 13 14:17:30 smithi055 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.a.service: Consumed 1.852s CPU time. 2024-02-13T14:17:31.369 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T14:17:31.559 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:32.754 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:32 smithi078 systemd[1]: Stopping Ceph grafana.a for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:33.007 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:32 smithi078 podman[44425]: 2024-02-13 14:17:32.75315317 +0000 UTC m=+0.122825532 container died 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, vcs-type=git, build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, release=239.1651231664, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, description=Ceph Grafana Container, distribution-scope=public, name=ubi8, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vendor=Red Hat, Inc., maintainer=Paul Cuzner , architecture=x86_64, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI) 2024-02-13T14:17:33.349 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 podman[44425]: 2024-02-13 14:17:33.122445054 +0000 UTC m=+0.492117413 container cleanup 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, io.buildah.version=1.32.0, com.redhat.component=ubi8-container, maintainer=Paul Cuzner , io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, release=239.1651231664, io.openshift.expose-services=, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, description=Ceph Grafana Container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, distribution-scope=public, io.k8s.display-name=Red Hat Universal Base Image 8, vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vcs-type=git, name=ubi8) 2024-02-13T14:17:33.349 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 bash[44425]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a 2024-02-13T14:17:33.687 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 podman[44444]: 2024-02-13 14:17:33.348298144 +0000 UTC m=+0.598173886 container remove 90454bf48f7489843795cc7f9666e5423df7b7993836330079c3bdbd171924e4 (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-grafana-a, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, distribution-scope=public, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, vcs-type=git, description=Ceph Grafana Container, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, maintainer=Paul Cuzner , version=8.5, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, io.openshift.tags=base rhel8, summary=Grafana Container configured for Ceph mgr/dashboard integration, com.redhat.component=ubi8-container, vendor=Red Hat, Inc., io.openshift.expose-services=, name=ubi8) 2024-02-13T14:17:33.687 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:17:33.687 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-13T14:17:33.687 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 systemd[1]: Stopped Ceph grafana.a for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:33.687 INFO:journalctl@ceph.grafana.a.smithi078.stdout:Feb 13 14:17:33 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@grafana.a.service: Consumed 2.737s CPU time. 2024-02-13T14:17:33.947 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:33 smithi078 systemd[1]: Stopping Ceph node-exporter.b for 9e367004-ca79-11ee-95b9-87774f69a715... 2024-02-13T14:17:34.209 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:33 smithi078 podman[44561]: 2024-02-13 14:17:33.94751762 +0000 UTC m=+0.127640158 container died e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:34.209 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 podman[44561]: 2024-02-13 14:17:34.209037931 +0000 UTC m=+0.389160520 container cleanup e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:34.460 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 bash[44561]: ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b 2024-02-13T14:17:34.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 podman[44580]: 2024-02-13 14:17:34.459871708 +0000 UTC m=+0.515002563 container remove e3da607aed60bcd835952f273bd2160ca06efe4b0842e4fa15106b9b6e17d36e (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-9e367004-ca79-11ee-95b9-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-13T14:17:34.778 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-13T14:17:34.779 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-13T14:17:34.779 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 systemd[1]: Stopped Ceph node-exporter.b for 9e367004-ca79-11ee-95b9-87774f69a715. 2024-02-13T14:17:34.779 INFO:journalctl@ceph.node-exporter.b.smithi078.stdout:Feb 13 14:17:34 smithi078 systemd[1]: ceph-9e367004-ca79-11ee-95b9-87774f69a715@node-exporter.b.service: Consumed 1.798s CPU time. 2024-02-13T14:17:35.273 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force --keep-logs 2024-02-13T14:17:35.464 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:36.596 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:17:36.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:17:36.666 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-13T14:17:36.696 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-13T14:17:36.697 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi055/crash 2024-02-13T14:17:36.698 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash -- . 2024-02-13T14:17:36.886 INFO:teuthology.orchestra.run.smithi055.stderr:tar: /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T14:17:36.886 INFO:teuthology.orchestra.run.smithi055.stderr:tar: Error is not recoverable: exiting now 2024-02-13T14:17:36.888 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi078/crash 2024-02-13T14:17:36.889 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash -- . 2024-02-13T14:17:36.943 INFO:teuthology.orchestra.run.smithi078.stderr:tar: /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T14:17:36.943 INFO:teuthology.orchestra.run.smithi078.stderr:tar: Error is not recoverable: exiting now 2024-02-13T14:17:36.945 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi170/crash 2024-02-13T14:17:36.946 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash -- . 2024-02-13T14:17:36.983 INFO:teuthology.orchestra.run.smithi170.stderr:tar: /var/lib/ceph/9e367004-ca79-11ee-95b9-87774f69a715/crash: Cannot open: No such file or directory 2024-02-13T14:17:36.983 INFO:teuthology.orchestra.run.smithi170.stderr:tar: Error is not recoverable: exiting now 2024-02-13T14:17:36.984 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-13T14:17:36.984 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-13T14:17:37.018 INFO:teuthology.orchestra.run.smithi055.stdout:2024-02-13T14:13:45.287104+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-13T14:17:37.018 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-13T14:17:37.018 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[SEC\]' /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-13T14:17:37.091 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[ERR\]' /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-13T14:17:37.163 DEBUG:teuthology.orchestra.run.smithi055:> sudo egrep '\[WRN\]' /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-13T14:17:37.234 INFO:teuthology.orchestra.run.smithi055.stdout:2024-02-13T14:13:45.287104+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-13T14:17:37.235 INFO:tasks.cephadm:Compressing logs... 2024-02-13T14:17:37.235 DEBUG:teuthology.orchestra.run.smithi055:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T14:17:37.278 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T14:17:37.280 DEBUG:teuthology.orchestra.run.smithi170:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-13T14:17:37.303 INFO:teuthology.orchestra.run.smithi055.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T14:17:37.304 INFO:teuthology.orchestra.run.smithi055.stderr:: No such file or directory 2024-02-13T14:17:37.304 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr: 57.8% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-13T14:17:37.306 INFO:teuthology.orchestra.run.smithi055.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T14:17:37.307 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log 2024-02-13T14:17:37.307 INFO:teuthology.orchestra.run.smithi055.stderr: 89.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T14:17:37.307 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T14:17:37.307 INFO:teuthology.orchestra.run.smithi055.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T14:17:37.307 INFO:teuthology.orchestra.run.smithi055.stderr: 82.7% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T14:17:37.308 INFO:teuthology.orchestra.run.smithi055.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log: 85.0% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:: No such file or directory 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.z.log 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T14:17:37.309 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log 2024-02-13T14:17:37.312 INFO:teuthology.orchestra.run.smithi170.stderr: 90.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T14:17:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:gzip/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log: -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log 2024-02-13T14:17:37.312 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr: 81.2% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- 89.5% /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr: 84.7% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log: 90.2% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.z.log.gz 2024-02-13T14:17:37.313 INFO:teuthology.orchestra.run.smithi170.stderr: 90.5% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T14:17:37.314 INFO:teuthology.orchestra.run.smithi078.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:: No such file or directory 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log: 89.5% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.audit.log.gz 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr: 81.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.cephadm.log.gz 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log: /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log: 84.5% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph.log.gz 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr: 81.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-13T14:17:37.315 INFO:teuthology.orchestra.run.smithi078.stderr: 90.5% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T14:17:37.316 INFO:teuthology.orchestra.run.smithi055.stderr: 90.4% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-volume.log.gz 2024-02-13T14:17:37.316 INFO:teuthology.orchestra.run.smithi078.stderr: 90.2% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.x.log.gz 2024-02-13T14:17:37.364 INFO:teuthology.orchestra.run.smithi170.stderr: 92.1% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.c.log.gz 2024-02-13T14:17:37.364 INFO:teuthology.orchestra.run.smithi078.stderr: 92.2% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.b.log.gz 2024-02-13T14:17:37.374 INFO:teuthology.orchestra.run.smithi055.stderr: 88.9% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mgr.y.log.gz 2024-02-13T14:17:37.483 INFO:teuthology.orchestra.run.smithi055.stderr: 91.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-mon.a.log.gz 2024-02-13T14:17:37.564 INFO:teuthology.orchestra.run.smithi170.stderr: 94.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.2.log.gz 2024-02-13T14:17:37.566 INFO:teuthology.orchestra.run.smithi170.stderr: 2024-02-13T14:17:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:real 0m0.272s 2024-02-13T14:17:37.566 INFO:teuthology.orchestra.run.smithi170.stderr:user 0m0.318s 2024-02-13T14:17:37.567 INFO:teuthology.orchestra.run.smithi170.stderr:sys 0m0.034s 2024-02-13T14:17:37.571 INFO:teuthology.orchestra.run.smithi055.stderr: 94.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.0.log.gz 2024-02-13T14:17:37.573 INFO:teuthology.orchestra.run.smithi055.stderr: 2024-02-13T14:17:37.573 INFO:teuthology.orchestra.run.smithi055.stderr:real 0m0.283s 2024-02-13T14:17:37.573 INFO:teuthology.orchestra.run.smithi055.stderr:user 0m0.519s 2024-02-13T14:17:37.573 INFO:teuthology.orchestra.run.smithi055.stderr:sys 0m0.037s 2024-02-13T14:17:37.613 INFO:teuthology.orchestra.run.smithi078.stderr: 94.3% -- replaced with /var/log/ceph/9e367004-ca79-11ee-95b9-87774f69a715/ceph-osd.1.log.gz 2024-02-13T14:17:37.615 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-02-13T14:17:37.615 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m0.320s 2024-02-13T14:17:37.615 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m0.367s 2024-02-13T14:17:37.615 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m0.036s 2024-02-13T14:17:37.615 INFO:tasks.cephadm:Archiving logs... 2024-02-13T14:17:37.615 DEBUG:teuthology.misc:Transferring archived files from smithi055:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi055/log 2024-02-13T14:17:37.616 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T14:17:37.740 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi078/log 2024-02-13T14:17:37.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T14:17:37.828 DEBUG:teuthology.misc:Transferring archived files from smithi170:/var/log/ceph to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi170/log 2024-02-13T14:17:37.829 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-13T14:17:37.910 INFO:tasks.cephadm:Removing cluster... 2024-02-13T14:17:37.911 DEBUG:teuthology.orchestra.run.smithi055:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force 2024-02-13T14:17:38.095 INFO:teuthology.orchestra.run.smithi055.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:38.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force 2024-02-13T14:17:38.392 INFO:teuthology.orchestra.run.smithi078.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:38.507 DEBUG:teuthology.orchestra.run.smithi170:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 9e367004-ca79-11ee-95b9-87774f69a715 --force 2024-02-13T14:17:38.694 INFO:teuthology.orchestra.run.smithi170.stdout:Deleting cluster with fsid: 9e367004-ca79-11ee-95b9-87774f69a715 2024-02-13T14:17:38.808 INFO:tasks.cephadm:Removing cephadm ... 2024-02-13T14:17:38.808 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-13T14:17:38.824 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-13T14:17:38.840 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-13T14:17:38.855 INFO:tasks.cephadm:Teardown complete 2024-02-13T14:17:38.855 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-13T14:17:38.865 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi055:/dev/vg_nvme/lv_1... 2024-02-13T14:17:38.865 DEBUG:teuthology.orchestra.run.smithi055:> sudo nvme disconnect -n lv_1 2024-02-13T14:17:39.000 INFO:teuthology.orchestra.run.smithi055.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-13T14:17:39.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.002 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi055:/dev/vg_nvme/lv_2... 2024-02-13T14:17:39.002 DEBUG:teuthology.orchestra.run.smithi055:> sudo nvme disconnect -n lv_2 2024-02-13T14:17:39.146 INFO:teuthology.orchestra.run.smithi055.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-13T14:17:39.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.148 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi055:/dev/vg_nvme/lv_3... 2024-02-13T14:17:39.148 DEBUG:teuthology.orchestra.run.smithi055:> sudo nvme disconnect -n lv_3 2024-02-13T14:17:39.302 INFO:teuthology.orchestra.run.smithi055.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-13T14:17:39.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.304 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi055:/dev/vg_nvme/lv_4... 2024-02-13T14:17:39.304 DEBUG:teuthology.orchestra.run.smithi055:> sudo nvme disconnect -n lv_4 2024-02-13T14:17:39.432 INFO:teuthology.orchestra.run.smithi055.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-13T14:17:39.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.434 DEBUG:teuthology.orchestra.run.smithi055:> set -ex 2024-02-13T14:17:39.434 DEBUG:teuthology.orchestra.run.smithi055:> sudo dd of=/scratch_devs 2024-02-13T14:17:39.461 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_1... 2024-02-13T14:17:39.461 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_1 2024-02-13T14:17:39.616 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-13T14:17:39.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.618 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_2... 2024-02-13T14:17:39.618 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_2 2024-02-13T14:17:39.757 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-13T14:17:39.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.759 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_3... 2024-02-13T14:17:39.760 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_3 2024-02-13T14:17:39.904 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-13T14:17:39.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:39.906 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi078:/dev/vg_nvme/lv_4... 2024-02-13T14:17:39.906 DEBUG:teuthology.orchestra.run.smithi078:> sudo nvme disconnect -n lv_4 2024-02-13T14:17:40.039 INFO:teuthology.orchestra.run.smithi078.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-13T14:17:40.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:40.041 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-02-13T14:17:40.041 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/scratch_devs 2024-02-13T14:17:40.068 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi170:/dev/vg_nvme/lv_1... 2024-02-13T14:17:40.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo nvme disconnect -n lv_1 2024-02-13T14:17:40.217 INFO:teuthology.orchestra.run.smithi170.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-13T14:17:40.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:40.218 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi170:/dev/vg_nvme/lv_2... 2024-02-13T14:17:40.219 DEBUG:teuthology.orchestra.run.smithi170:> sudo nvme disconnect -n lv_2 2024-02-13T14:17:40.368 INFO:teuthology.orchestra.run.smithi170.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-13T14:17:40.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:40.371 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi170:/dev/vg_nvme/lv_3... 2024-02-13T14:17:40.371 DEBUG:teuthology.orchestra.run.smithi170:> sudo nvme disconnect -n lv_3 2024-02-13T14:17:40.527 INFO:teuthology.orchestra.run.smithi170.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-13T14:17:40.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:40.528 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi170:/dev/vg_nvme/lv_4... 2024-02-13T14:17:40.528 DEBUG:teuthology.orchestra.run.smithi170:> sudo nvme disconnect -n lv_4 2024-02-13T14:17:40.666 INFO:teuthology.orchestra.run.smithi170.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-13T14:17:40.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:40.668 DEBUG:teuthology.orchestra.run.smithi170:> set -ex 2024-02-13T14:17:40.668 DEBUG:teuthology.orchestra.run.smithi170:> sudo dd of=/scratch_devs 2024-02-13T14:17:40.698 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-13T14:17:40.708 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-13T14:17:40.708 DEBUG:teuthology.orchestra.run.smithi055:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:17:40.711 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:17:40.713 DEBUG:teuthology.orchestra.run.smithi170:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-13T14:17:40.723 INFO:teuthology.orchestra.run.smithi055.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:17:40.727 INFO:teuthology.orchestra.run.smithi055.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:17:40.727 INFO:teuthology.orchestra.run.smithi055.stdout:=============================================================================== 2024-02-13T14:17:40.727 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 +853us[ +864us] +/- 84ms 2024-02-13T14:17:40.727 INFO:teuthology.orchestra.run.smithi055.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -802us[ -792us] +/- 67ms 2024-02-13T14:17:40.728 INFO:teuthology.orchestra.run.smithi055.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 57 +194us[ +204us] +/- 79ms 2024-02-13T14:17:40.728 INFO:teuthology.orchestra.run.smithi055.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:17:40.728 INFO:teuthology.orchestra.run.smithi078.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 +846us[ +862us] +/- 84ms 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 6 377 53 -795us[ -779us] +/- 67ms 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 +249us[ +265us] +/- 79ms 2024-02-13T14:17:40.732 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:17:40.754 INFO:teuthology.orchestra.run.smithi170.stderr:bash: line 1: ntpq: command not found 2024-02-13T14:17:40.758 INFO:teuthology.orchestra.run.smithi170.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-13T14:17:40.758 INFO:teuthology.orchestra.run.smithi170.stdout:=============================================================================== 2024-02-13T14:17:40.758 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 56 +846us[ +878us] +/- 84ms 2024-02-13T14:17:40.759 INFO:teuthology.orchestra.run.smithi170.stdout:^* hv02.front.sepia.ceph.com 3 6 377 54 -809us[ -776us] +/- 67ms 2024-02-13T14:17:40.759 INFO:teuthology.orchestra.run.smithi170.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 +192us[ +225us] +/- 79ms 2024-02-13T14:17:40.759 INFO:teuthology.orchestra.run.smithi170.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-13T14:17:40.759 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-13T14:17:40.769 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-13T14:17:40.769 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-13T14:17:40.778 DEBUG:teuthology.orchestra.run.smithi055:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-13T14:17:40.852 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-13T14:17:40.926 DEBUG:teuthology.orchestra.run.smithi170:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-13T14:17:41.000 DEBUG:teuthology.orchestra.run.smithi055:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:17:41.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:41.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:17:41.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:41.064 DEBUG:teuthology.orchestra.run.smithi170:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-13T14:17:41.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:41.096 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-13T14:17:41.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-13T14:17:41.128 INFO:teuthology.task.internal:Duration was 707.633089 seconds 2024-02-13T14:17:41.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-13T14:17:41.137 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-13T14:17:41.137 DEBUG:teuthology.orchestra.run.smithi055:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T14:17:41.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T14:17:41.142 DEBUG:teuthology.orchestra.run.smithi170:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-13T14:17:41.181 INFO:teuthology.orchestra.run.smithi055.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:17:41.184 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:17:41.187 INFO:teuthology.orchestra.run.smithi170.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-13T14:17:41.608 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-13T14:17:41.609 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi055.front.sepia.ceph.com 2024-02-13T14:17:41.609 DEBUG:teuthology.orchestra.run.smithi055:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T14:17:41.670 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-02-13T14:17:41.670 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T14:17:41.707 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi170.front.sepia.ceph.com 2024-02-13T14:17:41.707 DEBUG:teuthology.orchestra.run.smithi170:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-13T14:17:41.739 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-13T14:17:41.739 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T14:17:41.741 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T14:17:41.750 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-13T14:17:41.808 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-13T14:17:41.808 DEBUG:teuthology.orchestra.run.smithi055:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T14:17:41.811 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T14:17:41.821 DEBUG:teuthology.orchestra.run.smithi170:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-13T14:17:42.004 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-13T14:17:42.015 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-13T14:17:42.015 DEBUG:teuthology.orchestra.run.smithi055:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T14:17:42.042 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T14:17:42.069 DEBUG:teuthology.orchestra.run.smithi170:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-13T14:17:42.096 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-13T14:17:42.107 DEBUG:teuthology.orchestra.run.smithi055:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.112 DEBUG:teuthology.orchestra.run.smithi170:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.132 INFO:teuthology.orchestra.run.smithi055.stdout:kernel.core_pattern = core 2024-02-13T14:17:42.134 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-02-13T14:17:42.162 INFO:teuthology.orchestra.run.smithi170.stdout:kernel.core_pattern = core 2024-02-13T14:17:42.182 DEBUG:teuthology.orchestra.run.smithi055:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:42.211 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:42.225 DEBUG:teuthology.orchestra.run.smithi170:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-13T14:17:42.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-13T14:17:42.239 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-13T14:17:42.250 INFO:teuthology.task.internal:Transferring archived files... 2024-02-13T14:17:42.251 DEBUG:teuthology.misc:Transferring archived files from smithi055:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi055 2024-02-13T14:17:42.251 DEBUG:teuthology.orchestra.run.smithi055:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T14:17:42.288 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi078 2024-02-13T14:17:42.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T14:17:42.325 DEBUG:teuthology.misc:Transferring archived files from smithi170:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021/remote/smithi170 2024-02-13T14:17:42.325 DEBUG:teuthology.orchestra.run.smithi170:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-13T14:17:42.362 INFO:teuthology.task.internal:Removing archive directory... 2024-02-13T14:17:42.362 DEBUG:teuthology.orchestra.run.smithi055:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T14:17:42.365 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T14:17:42.367 DEBUG:teuthology.orchestra.run.smithi170:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-13T14:17:42.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-13T14:17:42.422 INFO:teuthology.task.internal:Not uploading archives. 2024-02-13T14:17:42.422 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-13T14:17:42.431 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-13T14:17:42.432 DEBUG:teuthology.orchestra.run.smithi055:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T14:17:42.434 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T14:17:42.436 DEBUG:teuthology.orchestra.run.smithi170:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-13T14:17:42.445 INFO:teuthology.orchestra.run.smithi055.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 14:17 /home/ubuntu/cephtest 2024-02-13T14:17:42.448 INFO:teuthology.orchestra.run.smithi078.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 14:17 /home/ubuntu/cephtest 2024-02-13T14:17:42.464 INFO:teuthology.orchestra.run.smithi170.stdout: 265293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 13 14:17 /home/ubuntu/cephtest 2024-02-13T14:17:42.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-13T14:17:42.476 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-13T14:17:42.497 INFO:teuthology.nuke:Checking targets against current locks 2024-02-13T14:17:42.523 DEBUG:teuthology.nuke:shortname: smithi055 2024-02-13T14:17:42.523 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:17:42.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi055.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.734123', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMYaR4eqleWPd3FOSNAoHSp4DmjLsuRLWUoySUWyHHtGDGN9bf9BFUKPQiNuoWOOrBvSz8N+z+znsbGLiv9D31Y='} 2024-02-13T14:17:42.549 DEBUG:teuthology.nuke:shortname: smithi078 2024-02-13T14:17:42.549 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:17:42.572 DEBUG:teuthology.nuke:shortname: smithi170 2024-02-13T14:17:42.572 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-13T14:17:42.575 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.735323', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLH6V8YJUcEN3NKh6RpRi6+fLcTt689WKZ7rQ0Ja4eYgguJG3wfyRt1PaYkNyj6NN3LOzAqTLTwXEojiA81Zi24='} 2024-02-13T14:17:42.588 INFO:teuthology.orchestra.console.smithi055:Power off 2024-02-13T14:17:42.589 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T14:17:42.601 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi170.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/phlogistonjohn-2024-02-13_11:41:43-orch:cephadm-wip-phlogistonjohn-testing-2024-02-12-1654-distro-default-smithi/7558021', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-13 13:57:40.732770', 'locked_by': 'scheduled_phlogistonjohn@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCNvjbm4hiy1kHXarPlsBzAe0C81UUSOvfK5nP94FfcfbqbWXTIWjmFj8PHGcDBPLKRDY3Drzhga22MMQ7ULxFM='} 2024-02-13T14:17:42.610 DEBUG:teuthology.orchestra.console.smithi055:power off output: Chassis Power Control: Down/Off 2024-02-13T14:17:42.610 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:42.628 INFO:teuthology.orchestra.console.smithi078:Power off 2024-02-13T14:17:42.629 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T14:17:42.642 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-02-13T14:17:42.651 DEBUG:teuthology.orchestra.console.smithi078:power off output: Chassis Power Control: Down/Off 2024-02-13T14:17:42.652 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:42.670 INFO:teuthology.orchestra.console.smithi170:Power off 2024-02-13T14:17:42.670 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-13T14:17:42.679 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-02-13T14:17:42.693 DEBUG:teuthology.orchestra.console.smithi170:power off output: Chassis Power Control: Down/Off 2024-02-13T14:17:42.693 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:42.715 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-02-13T14:17:46.644 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:46.680 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:46.716 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:46.753 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is on 2024-02-13T14:17:46.789 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is on 2024-02-13T14:17:46.826 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is on 2024-02-13T14:17:50.756 DEBUG:teuthology.orchestra.console.smithi055:pexpect command: ipmitool -H smithi055.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:50.790 DEBUG:teuthology.orchestra.console.smithi078:pexpect command: ipmitool -H smithi078.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:50.827 DEBUG:teuthology.orchestra.console.smithi170:pexpect command: ipmitool -H smithi170.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-13T14:17:50.865 DEBUG:teuthology.orchestra.console.smithi055:check power output: Chassis Power is off 2024-02-13T14:17:50.898 DEBUG:teuthology.orchestra.console.smithi078:check power output: Chassis Power is off 2024-02-13T14:17:50.936 DEBUG:teuthology.orchestra.console.smithi170:check power output: Chassis Power is off 2024-02-13T14:17:50.967 INFO:teuthology.orchestra.console.smithi055:Power off completed 2024-02-13T14:17:50.999 INFO:teuthology.orchestra.console.smithi078:Power off completed 2024-02-13T14:17:51.037 INFO:teuthology.orchestra.console.smithi170:Power off completed 2024-02-13T14:17:51.140 INFO:teuthology.run:Summary data: description: orch:cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/classic start} duration: 707.6330890655518 failure_reason: '"2024-02-13T14:13:45.287104+0000 mon.a (mon.0) 189 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_phlogistonjohn@teuthology success: false 2024-02-13T14:17:51.141 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-13T14:17:51.213 INFO:teuthology.run:FAIL