2024-02-10T07:29:57.004 INFO:root:teuthology version: 0.0.1.dev256+gd9fdb22 2024-02-10T07:29:57.004 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222 --verbose --description orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} --name yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/orig.config.yaml 2024-02-10T07:29:57.032 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T07:29:57.113 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222 branch: main description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} email: yweinste@redhat.com first_in_suite: false job_id: '7555222' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 1 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mgr/cephadm/use_agent: false mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: main sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - osd.0 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.1 - client.1 - prometheus.a - grafana.a - node-exporter.b - - mon.c - mgr.z - osd.2 - client.2 - node-exporter.c seed: 1610 sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 sleep_before_teardown: 0 subset: 111/120000 suite: orch suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 targets: smithi022.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEyxbFM6rlK+NYUbKVSETSKWLrwjA3qkXVbObPyHOnouwTftuSdmzbDWgXVZPH+MYXYvFgyKLldt5AiJQdPYZk= smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0r0xdU/X2dzcRgIL95qYCP2Kz7TjiODmLwlI2lMgH1G0RhiWs2Yb8pVVUQamwDZNID6qUTpMjEkkv/WxC7uIE= smithi081.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSdWICprRMhyy/1rUMIH2J1gYde9QewfmaduvVjk6RJn+kjD39f9CsuesEegECZlg9rRBfeewfRrMCXimVGGnA= tasks: - pexec: all: - sudo dnf install runc -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - nvme_loop: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - cephadm.shell: mon.a: - stat -c '%u %g' /var/log/ceph | grep '167 167' - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - ceph orch ls --format yaml - ceph orch ls | grep '^osd ' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: d9fdb2209e15b39d9f061fd85399f352ce0f0894 timestamp: 2024-02-09_19:44:16 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-10T07:29:57.113 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa; will attempt to use it 2024-02-10T07:29:57.114 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/qa/tasks 2024-02-10T07:29:57.114 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-10T07:29:57.116 INFO:teuthology.task.internal:Checking packages... 2024-02-10T07:29:57.140 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90' 2024-02-10T07:29:57.140 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-10T07:29:57.140 INFO:teuthology.packaging:ref: None 2024-02-10T07:29:57.140 INFO:teuthology.packaging:tag: None 2024-02-10T07:29:57.140 INFO:teuthology.packaging:branch: main 2024-02-10T07:29:57.141 INFO:teuthology.packaging:sha1: 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:29:57.141 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=main 2024-02-10T07:29:57.405 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1361.gf8e4e331 2024-02-10T07:29:57.406 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-10T07:29:57.413 INFO:teuthology.task.internal:no buildpackages task found 2024-02-10T07:29:57.413 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-10T07:29:57.442 INFO:teuthology.task.internal:Saving configuration 2024-02-10T07:29:57.454 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-10T07:29:57.460 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:29:57.483 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.163470', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEyxbFM6rlK+NYUbKVSETSKWLrwjA3qkXVbObPyHOnouwTftuSdmzbDWgXVZPH+MYXYvFgyKLldt5AiJQdPYZk='} 2024-02-10T07:29:57.505 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.162075', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0r0xdU/X2dzcRgIL95qYCP2Kz7TjiODmLwlI2lMgH1G0RhiWs2Yb8pVVUQamwDZNID6qUTpMjEkkv/WxC7uIE='} 2024-02-10T07:29:57.526 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.160175', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSdWICprRMhyy/1rUMIH2J1gYde9QewfmaduvVjk6RJn+kjD39f9CsuesEegECZlg9rRBfeewfRrMCXimVGGnA='} 2024-02-10T07:29:57.526 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-10T07:29:57.532 INFO:teuthology.task.internal:roles: ubuntu@smithi022.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-10T07:29:57.532 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.1', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b'] 2024-02-10T07:29:57.532 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'osd.2', 'client.2', 'node-exporter.c'] 2024-02-10T07:29:57.532 INFO:teuthology.run_tasks:Running task console_log... 2024-02-10T07:29:57.668 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f787718e160>, signals=[15]) 2024-02-10T07:29:57.669 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-10T07:29:57.675 INFO:teuthology.task.internal:Opening connections... 2024-02-10T07:29:57.675 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi022.front.sepia.ceph.com 2024-02-10T07:29:57.677 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:29:57.799 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2024-02-10T07:29:57.800 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:29:57.903 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2024-02-10T07:29:57.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:29:57.993 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-10T07:29:57.999 DEBUG:teuthology.orchestra.run.smithi022:> uname -m 2024-02-10T07:29:58.018 INFO:teuthology.orchestra.run.smithi022.stdout:x86_64 2024-02-10T07:29:58.018 DEBUG:teuthology.orchestra.run.smithi022:> cat /etc/os-release 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:NAME="CentOS Stream" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION="9" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:ID="centos" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:ID_LIKE="rhel fedora" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:VERSION_ID="9" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:PLATFORM_ID="platform:el9" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:ANSI_COLOR="0;31" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:LOGO="fedora-logo-icon" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:HOME_URL="https://centos.org/" 2024-02-10T07:29:58.076 INFO:teuthology.orchestra.run.smithi022.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T07:29:58.077 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T07:29:58.077 INFO:teuthology.orchestra.run.smithi022.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T07:29:58.077 INFO:teuthology.lock.ops:Updating smithi022.front.sepia.ceph.com on lock server 2024-02-10T07:29:58.101 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2024-02-10T07:29:58.119 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2024-02-10T07:29:58.119 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T07:29:58.177 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2024-02-10T07:29:58.178 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T07:29:58.178 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T07:29:58.178 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T07:29:58.178 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2024-02-10T07:29:58.201 DEBUG:teuthology.orchestra.run.smithi081:> uname -m 2024-02-10T07:29:58.219 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2024-02-10T07:29:58.219 DEBUG:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2024-02-10T07:29:58.275 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Stream" 2024-02-10T07:29:58.275 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:PLATFORM_ID="platform:el9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Stream 9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:LOGO="fedora-logo-icon" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://centos.org/" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-02-10T07:29:58.276 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-02-10T07:29:58.277 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2024-02-10T07:29:58.300 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-10T07:29:58.310 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-10T07:29:58.317 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-10T07:29:58.318 DEBUG:teuthology.orchestra.run.smithi022:> test '!' -e /home/ubuntu/cephtest 2024-02-10T07:29:58.320 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2024-02-10T07:29:58.323 DEBUG:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2024-02-10T07:29:58.338 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-10T07:29:58.344 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-10T07:29:58.344 DEBUG:teuthology.orchestra.run.smithi022:> test -z $(ls -A /var/lib/ceph) 2024-02-10T07:29:58.378 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2024-02-10T07:29:58.380 DEBUG:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2024-02-10T07:29:58.482 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-10T07:29:58.562 INFO:teuthology.run_tasks:Running task kernel... 2024-02-10T07:29:58.576 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-10T07:29:58.576 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}, 'mon.c': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-10T07:29:58.579 DEBUG:teuthology.orchestra.run.smithi022:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T07:29:58.580 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T07:29:58.580 DEBUG:teuthology.orchestra.run.smithi081:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-10T07:29:58.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:29:58.597 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-02-10T07:29:58.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:29:58.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:29:58.599 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-02-10T07:29:58.599 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-02-10T07:29:58.655 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-361.el9.x86_64 2024-02-10T07:29:58.655 INFO:teuthology.task.kernel:Running kernel on smithi081: 5.14.0-361.el9.x86_64 2024-02-10T07:29:58.655 DEBUG:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2024-02-10T07:29:58.656 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-361.el9.x86_64 2024-02-10T07:29:58.657 INFO:teuthology.task.kernel:Running kernel on smithi022: 5.14.0-361.el9.x86_64 2024-02-10T07:29:58.657 DEBUG:teuthology.orchestra.run.smithi022:> sudo yum install -y kernel 2024-02-10T07:29:58.658 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-361.el9.x86_64 2024-02-10T07:29:58.658 INFO:teuthology.task.kernel:Running kernel on smithi027: 5.14.0-361.el9.x86_64 2024-02-10T07:29:58.658 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2024-02-10T07:30:00.858 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-10T07:30:00.885 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - AppStream 13 MB/s | 6.7 MB 00:00 2024-02-10T07:30:00.895 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-02-10T07:30:02.289 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 7.5 MB/s | 2.0 MB 00:00 2024-02-10T07:30:02.344 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS-9 - BaseOS 7.4 MB/s | 2.0 MB 00:00 2024-02-10T07:30:02.504 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS-9 - BaseOS 4.1 MB/s | 2.0 MB 00:00 2024-02-10T07:30:03.288 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-02-10T07:30:04.403 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - BaseOS 5.1 MB/s | 8.0 MB 00:01 2024-02-10T07:30:05.139 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 25 MB/s | 19 MB 00:00 2024-02-10T07:30:09.020 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - AppStream 5.4 MB/s | 19 MB 00:03 2024-02-10T07:30:10.582 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.9 MB 00:01 2024-02-10T07:30:12.249 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2024-02-10T07:30:15.294 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 7.0 MB/s | 20 MB 00:02 2024-02-10T07:30:15.854 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 5.9 MB 00:03 2024-02-10T07:30:17.238 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 15 kB 00:00 2024-02-10T07:30:20.366 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - BaseOS 464 kB/s | 8.0 MB 00:17 2024-02-10T07:30:20.400 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-02-10T07:30:22.766 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T07:30:22.766 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T07:30:22.866 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-02-10T07:30:22.871 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:30:22.871 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2024-02-10T07:30:22.871 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:30:22.871 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-02-10T07:30:22.871 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout:Install 4 Packages 2024-02-10T07:30:22.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:30:22.873 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 99 M 2024-02-10T07:30:22.873 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 122 M 2024-02-10T07:30:22.873 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-02-10T07:30:23.627 INFO:teuthology.orchestra.run.smithi027.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 8.0 MB/s | 6.0 MB 00:00 2024-02-10T07:30:24.252 INFO:teuthology.orchestra.run.smithi027.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 15 MB/s | 21 MB 00:01 2024-02-10T07:30:24.853 INFO:teuthology.orchestra.run.smithi027.stdout:(3/4): kernel-modules-core-5.14.0-419.el9.x86_6 27 MB/s | 33 MB 00:01 2024-02-10T07:30:25.319 INFO:teuthology.orchestra.run.smithi027.stdout:(4/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 16 MB/s | 39 MB 00:02 2024-02-10T07:30:25.323 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:25.323 INFO:teuthology.orchestra.run.smithi027.stdout:Total 41 MB/s | 99 MB 00:02 2024-02-10T07:30:25.323 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-02-10T07:30:25.771 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-02-10T07:30:25.771 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-02-10T07:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-02-10T07:30:26.832 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-02-10T07:30:29.351 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-02-10T07:30:30.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:30:30.212 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:30:33.129 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:30:33.155 INFO:teuthology.orchestra.run.smithi081.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 20 MB 00:15 2024-02-10T07:30:33.307 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:30:38.138 INFO:teuthology.orchestra.run.smithi081.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-02-10T07:30:38.365 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:30:38.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:30:40.412 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T07:30:40.412 INFO:teuthology.orchestra.run.smithi081.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T07:30:40.511 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-02-10T07:30:40.519 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:30:40.519 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2024-02-10T07:30:40.519 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout:Installing dependencies: 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout:Install 4 Packages 2024-02-10T07:30:40.520 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:30:40.521 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 99 M 2024-02-10T07:30:40.521 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 122 M 2024-02-10T07:30:40.521 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-02-10T07:30:40.873 INFO:teuthology.orchestra.run.smithi081.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 17 MB/s | 6.0 MB 00:00 2024-02-10T07:30:41.590 INFO:teuthology.orchestra.run.smithi081.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 19 MB/s | 21 MB 00:01 2024-02-10T07:30:41.974 INFO:teuthology.orchestra.run.smithi081.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 27 MB/s | 39 MB 00:01 2024-02-10T07:30:42.432 INFO:teuthology.orchestra.run.smithi081.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 21 MB/s | 33 MB 00:01 2024-02-10T07:30:42.435 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:30:42.435 INFO:teuthology.orchestra.run.smithi081.stdout:Total 52 MB/s | 99 MB 00:01 2024-02-10T07:30:42.435 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-02-10T07:30:42.809 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-02-10T07:30:42.809 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-02-10T07:30:43.560 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:30:43.865 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-02-10T07:30:43.866 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-02-10T07:30:46.269 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-02-10T07:30:46.992 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:30:47.125 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:30:49.876 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:30:50.063 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:30:54.786 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:30:55.016 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:30:59.760 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:14.591 INFO:teuthology.orchestra.run.smithi027.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T07:31:14.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:31:14.662 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:15.163 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:15.163 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:31:15.163 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:31:15.163 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T07:31:15.603 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:31:15.604 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-02-10T07:31:16.145 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:31:16.284 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:16.284 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:31:16.284 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T07:31:16.285 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T07:31:16.285 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T07:31:16.285 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-10T07:31:16.285 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T07:31:16.285 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:31:16.396 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:16.397 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:31:16.397 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2024-02-10T07:31:16.443 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2024-02-10T07:31:16.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:31:16.445 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T07:31:16.445 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T07:31:16.445 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 5.14.0-419.el9.x86_64 2024-02-10T07:31:16.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T07:31:17.006 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2024-02-10T07:31:18.511 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T07:31:18.542 INFO:teuthology.orchestra.run.smithi027.stderr:done 2024-02-10T07:31:18.544 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2024-02-10T07:31:18.559 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.9Hl9lXbo5s 2024-02-10T07:31:18.559 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.9Hl9lXbo5s 2024-02-10T07:31:18.631 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.9Hl9lXbo5s 2024-02-10T07:31:18.773 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.9Hl9lXbo5s is 6KB 2024-02-10T07:31:18.820 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.9Hl9lXbo5s 2024-02-10T07:31:18.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T07:31:18.903 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T07:31:18.903 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T07:31:18.903 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T07:31:18.903 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T07:31:18.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T07:31:19.065 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2024-02-10T07:31:29.162 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:29.162 INFO:teuthology.orchestra.run.smithi081.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T07:31:29.162 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:31:29.232 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:29.574 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - AppStream 282 kB/s | 19 MB 01:08 2024-02-10T07:31:29.626 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:29.626 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:31:29.626 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:31:29.626 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.039 INFO:teuthology.orchestra.run.smithi081.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.040 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:31:30.040 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-02-10T07:31:30.523 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:31:30.654 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.654 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:31:30.654 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T07:31:30.654 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T07:31:30.654 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T07:31:30.654 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-02-10T07:31:30.654 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T07:31:30.655 DEBUG:teuthology.orchestra.run.smithi081:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:31:30.763 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:31:30.763 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:31:30.763 DEBUG:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2-tools 2024-02-10T07:31:30.805 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2-tools 2024-02-10T07:31:30.805 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2024-02-10T07:31:30.805 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.06 2024-02-10T07:31:30.805 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 68.el9 2024-02-10T07:31:30.805 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Group : Unspecified 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 8264254 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : builder@centos.org 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Support tools for GRUB. 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2024-02-10T07:31:30.806 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:31:30.807 INFO:teuthology.orchestra.run.smithi081.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T07:31:30.808 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T07:31:30.808 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 5.14.0-419.el9.x86_64 2024-02-10T07:31:30.808 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T07:31:31.435 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2024-02-10T07:31:32.869 INFO:teuthology.orchestra.run.smithi081.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T07:31:32.903 INFO:teuthology.orchestra.run.smithi081.stderr:done 2024-02-10T07:31:32.910 DEBUG:teuthology.orchestra.run.smithi081:> mktemp 2024-02-10T07:31:32.928 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.eoy1QQ7QlZ 2024-02-10T07:31:32.928 DEBUG:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.eoy1QQ7QlZ 2024-02-10T07:31:33.000 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.eoy1QQ7QlZ 2024-02-10T07:31:33.141 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.eoy1QQ7QlZ is 6KB 2024-02-10T07:31:33.191 DEBUG:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.eoy1QQ7QlZ 2024-02-10T07:31:33.205 DEBUG:teuthology.orchestra.run.smithi081:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T07:31:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T07:31:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T07:31:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T07:31:33.270 INFO:teuthology.orchestra.run.smithi081.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T07:31:33.272 DEBUG:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T07:31:33.429 DEBUG:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2024-02-10T07:31:33.591 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - CRB 16 MB/s | 5.9 MB 00:00 2024-02-10T07:31:35.226 INFO:teuthology.orchestra.run.smithi022.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 15 kB 00:00 2024-02-10T07:31:36.935 INFO:teuthology.orchestra.run.smithi022.stdout:Extra Packages for Enterprise Linux 13 MB/s | 20 MB 00:01 2024-02-10T07:31:41.960 INFO:teuthology.orchestra.run.smithi022.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-02-10T07:31:44.359 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-02-10T07:31:44.359 INFO:teuthology.orchestra.run.smithi022.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-02-10T07:31:44.460 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout: Package Arch Version Repository Size 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout: kernel x86_64 5.14.0-419.el9 CentOS-BaseOS 6.0 M 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout:Installing dependencies: 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core x86_64 5.14.0-419.el9 CentOS-BaseOS 21 M 2024-02-10T07:31:44.472 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules x86_64 5.14.0-419.el9 CentOS-BaseOS 39 M 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core x86_64 5.14.0-419.el9 CentOS-BaseOS 33 M 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:Install 4 Packages 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 99 M 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 122 M 2024-02-10T07:31:44.473 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-10T07:31:44.936 INFO:teuthology.orchestra.run.smithi022.stdout:(1/4): kernel-5.14.0-419.el9.x86_64.rpm 13 MB/s | 6.0 MB 00:00 2024-02-10T07:31:45.303 INFO:teuthology.orchestra.run.smithi022.stdout:(2/4): kernel-core-5.14.0-419.el9.x86_64.rpm 25 MB/s | 21 MB 00:00 2024-02-10T07:31:45.870 INFO:teuthology.orchestra.run.smithi022.stdout:(3/4): kernel-modules-5.14.0-419.el9.x86_64.rpm 28 MB/s | 39 MB 00:01 2024-02-10T07:31:46.378 INFO:teuthology.orchestra.run.smithi022.stdout:(4/4): kernel-modules-core-5.14.0-419.el9.x86_6 23 MB/s | 33 MB 00:01 2024-02-10T07:31:46.382 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:31:46.382 INFO:teuthology.orchestra.run.smithi022.stdout:Total 52 MB/s | 99 MB 00:01 2024-02-10T07:31:46.382 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-10T07:31:46.832 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-10T07:31:46.835 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-10T07:31:47.893 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-10T07:31:47.894 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-10T07:31:49.068 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-10T07:31:49.069 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:31:49.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:31:50.463 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-10T07:31:51.140 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-core-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:31:51.286 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:31:54.188 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:31:54.373 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:31:59.429 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:31:59.538 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:03.435 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-02-10T07:32:03.435 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:32:03.436 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:32:04.616 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:07.612 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-10T07:32:16.613 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:32:16.614 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:32:19.676 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-10T07:32:21.948 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-02-10T07:32:30.949 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:32:30.950 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:32:31.677 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:32:31.678 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:32:34.012 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-02-10T07:32:34.748 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-10T07:32:35.060 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:35.060 INFO:teuthology.orchestra.run.smithi022.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-02-10T07:32:35.060 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:32:35.131 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-modules-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:35.565 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: kernel-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:35.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-5.14.0-419.el9.x86_64 1/4 2024-02-10T07:32:35.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-core-5.14.0-419.el9.x86_64 2/4 2024-02-10T07:32:35.565 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-5.14.0-419.el9.x86_64 3/4 2024-02-10T07:32:36.020 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : kernel-modules-core-5.14.0-419.el9.x86_64 4/4 2024-02-10T07:32:36.020 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:32:36.020 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-10T07:32:36.020 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.021 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-core-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.021 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.021 INFO:teuthology.orchestra.run.smithi022.stdout: kernel-modules-core-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.021 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:32:36.021 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-10T07:32:36.524 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:32:36.658 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.659 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:32:36.659 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-419.el9.x86_64 2024-02-10T07:32:36.659 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-10T07:32:36.659 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-10T07:32:36.659 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-10T07:32:36.659 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi022.front.sepia.ceph.com, path=None, version=distro) 2024-02-10T07:32:36.659 DEBUG:teuthology.orchestra.run.smithi022:> rpm -q kernel | sort -rV | head -n 1 2024-02-10T07:32:36.771 INFO:teuthology.orchestra.run.smithi022.stdout:kernel-5.14.0-419.el9.x86_64 2024-02-10T07:32:36.771 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-419.el9.x86_64 2024-02-10T07:32:36.772 DEBUG:teuthology.orchestra.run.smithi022:> sudo rpm -qi grub2-tools 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Name : grub2-tools 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Epoch : 1 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Version : 2.06 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Release : 68.el9 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Architecture: x86_64 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-02-10T07:32:36.809 INFO:teuthology.orchestra.run.smithi022.stdout:Group : Unspecified 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Size : 8264254 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:License : GPLv3+ 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Packager : builder@centos.org 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Vendor : CentOS 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:URL : http://www.gnu.org/software/grub/ 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Summary : Support tools for GRUB. 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout:Description : 2024-02-10T07:32:36.810 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout:customizable bootloader with modular architecture. It supports a rich 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout:variety of kernel formats, file systems, computer architectures and 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout:hardware devices. 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:32:36.811 INFO:teuthology.orchestra.run.smithi022.stdout:This subpackage provides tools for support of all platforms. 2024-02-10T07:32:36.812 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-02-10T07:32:36.812 INFO:teuthology.task.kernel:Updating grub on smithi022 to boot 5.14.0-419.el9.x86_64 2024-02-10T07:32:36.812 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-02-10T07:32:37.403 INFO:teuthology.orchestra.run.smithi022.stderr:Generating grub configuration file ... 2024-02-10T07:32:38.891 INFO:teuthology.orchestra.run.smithi022.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-02-10T07:32:38.925 INFO:teuthology.orchestra.run.smithi022.stderr:done 2024-02-10T07:32:38.928 DEBUG:teuthology.orchestra.run.smithi022:> mktemp 2024-02-10T07:32:38.943 INFO:teuthology.orchestra.run.smithi022.stdout:/tmp/tmp.uOH1e73NkL 2024-02-10T07:32:38.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uOH1e73NkL 2024-02-10T07:32:39.019 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0666 /tmp/tmp.uOH1e73NkL 2024-02-10T07:32:39.169 DEBUG:teuthology.orchestra.remote:smithi022:/tmp/tmp.uOH1e73NkL is 6KB 2024-02-10T07:32:39.217 DEBUG:teuthology.orchestra.run.smithi022:> rm -fr /tmp/tmp.uOH1e73NkL 2024-02-10T07:32:39.232 DEBUG:teuthology.orchestra.run.smithi022:> sudo /bin/ls /boot/loader/entries || true 2024-02-10T07:32:39.300 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-02-10T07:32:39.300 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-02-10T07:32:39.300 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-02-10T07:32:39.301 INFO:teuthology.orchestra.run.smithi022.stdout:15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64.conf 2024-02-10T07:32:39.302 DEBUG:teuthology.orchestra.run.smithi022:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-419.el9.x86_64 2024-02-10T07:32:39.465 DEBUG:teuthology.orchestra.run.smithi022:> sudo shutdown -r now 2024-02-10T07:32:46.017 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:32:46.018 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:32:49.752 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:32:49.752 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:33:04.444 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.81 2024-02-10T07:33:08.288 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2024-02-10T07:33:09.468 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-10T07:33:09.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:33:09.469 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:33:19.452 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:33:19.453 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:33:19.868 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-02-10T07:33:20.274 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:33:20.274 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.14.0-419.el9.x86_64"... 2024-02-10T07:33:20.274 DEBUG:teuthology.orchestra.run.smithi081:> uname -r 2024-02-10T07:33:20.288 INFO:teuthology.orchestra.run.smithi081.stdout:5.14.0-419.el9.x86_64 2024-02-10T07:33:20.288 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T07:33:20.288 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T07:33:20.288 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T07:33:21.289 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-02-10T07:33:21.290 DEBUG:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T07:33:21.369 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2024-02-10T07:33:21.411 DEBUG:teuthology.parallel:result is None 2024-02-10T07:33:26.291 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:33:26.291 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:33:26.693 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-02-10T07:33:27.129 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:33:27.129 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-419.el9.x86_64"... 2024-02-10T07:33:27.129 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2024-02-10T07:33:27.144 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-419.el9.x86_64 2024-02-10T07:33:27.144 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T07:33:27.144 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T07:33:27.144 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T07:33:27.996 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.22 2024-02-10T07:33:28.145 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-10T07:33:28.146 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T07:33:28.214 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2024-02-10T07:33:28.260 DEBUG:teuthology.parallel:result is None 2024-02-10T07:33:36.997 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:33:36.998 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:34:37.001 DEBUG:teuthology.orchestra.remote:timed out 2024-02-10T07:34:49.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:34:49.004 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:34:49.400 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-02-10T07:34:49.791 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:34:49.791 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-419.el9.x86_64"... 2024-02-10T07:34:49.791 DEBUG:teuthology.orchestra.run.smithi022:> uname -r 2024-02-10T07:34:49.810 INFO:teuthology.orchestra.run.smithi022.stdout:5.14.0-419.el9.x86_64 2024-02-10T07:34:49.811 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-419.el9.x86_64 vs 5.14.0-419.el9.x86_64 2024-02-10T07:34:49.811 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-10T07:34:49.811 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-02-10T07:34:50.812 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-10T07:34:50.813 DEBUG:teuthology.orchestra.run.smithi022:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-10T07:34:50.882 INFO:teuthology.orchestra.run.smithi022.stdout:ttyS1 2024-02-10T07:34:50.917 DEBUG:teuthology.parallel:result is None 2024-02-10T07:34:50.917 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-10T07:34:50.925 INFO:teuthology.task.internal:Creating test directory... 2024-02-10T07:34:50.925 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T07:34:50.927 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T07:34:50.931 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-10T07:34:50.950 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-10T07:34:50.957 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-10T07:34:50.964 INFO:teuthology.task.internal:Creating archive directory... 2024-02-10T07:34:50.965 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T07:34:50.986 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T07:34:50.990 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-10T07:34:51.080 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-10T07:34:51.086 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-10T07:34:51.086 DEBUG:teuthology.orchestra.run.smithi022:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T07:34:51.089 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T07:34:51.091 DEBUG:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-10T07:34:51.134 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.135 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.148 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.148 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.153 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.163 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-10T07:34:51.165 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-10T07:34:51.171 INFO:teuthology.task.internal:Configuring sudo... 2024-02-10T07:34:51.172 DEBUG:teuthology.orchestra.run.smithi022:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T07:34:51.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T07:34:51.178 DEBUG:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-10T07:34:51.209 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-10T07:34:51.218 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-10T07:34:51.219 DEBUG:teuthology.orchestra.run.smithi022:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T07:34:51.243 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T07:34:51.252 DEBUG:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-10T07:34:51.268 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.394 DEBUG:teuthology.orchestra.run.smithi022:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:51.476 DEBUG:teuthology.orchestra.run.smithi022:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:51.520 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:34:51.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T07:34:51.587 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.695 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:51.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:51.789 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:34:51.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T07:34:51.862 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.904 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-10T07:34:51.963 DEBUG:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:52.043 DEBUG:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-10T07:34:52.094 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:34:52.094 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-10T07:34:52.165 DEBUG:teuthology.orchestra.run.smithi022:> sudo service rsyslog restart 2024-02-10T07:34:52.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2024-02-10T07:34:52.171 DEBUG:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2024-02-10T07:34:52.214 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:34:52.231 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:34:52.265 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:34:52.658 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-10T07:34:52.667 INFO:teuthology.task.internal:Starting timer... 2024-02-10T07:34:52.667 INFO:teuthology.run_tasks:Running task pcp... 2024-02-10T07:34:52.678 INFO:teuthology.run_tasks:Running task selinux... 2024-02-10T07:34:52.687 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-02-10T07:34:52.688 DEBUG:teuthology.orchestra.run.smithi022:> sudo service auditd rotate 2024-02-10T07:34:52.751 INFO:teuthology.orchestra.run.smithi022.stdout:Rotating logs: 2024-02-10T07:34:52.754 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2024-02-10T07:34:52.837 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2024-02-10T07:34:52.839 DEBUG:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2024-02-10T07:34:52.893 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: 2024-02-10T07:34:52.895 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-10T07:34:52.896 DEBUG:teuthology.orchestra.run.smithi022:> /usr/sbin/getenforce 2024-02-10T07:34:52.921 INFO:teuthology.orchestra.run.smithi022.stdout:Permissive 2024-02-10T07:34:52.921 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2024-02-10T07:34:52.943 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2024-02-10T07:34:52.943 DEBUG:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2024-02-10T07:34:52.970 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2024-02-10T07:34:52.971 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi022.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2024-02-10T07:34:52.971 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:34:52.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:34:52.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:34:53.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:34:53.034 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:34:53.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:34:53.066 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-10T07:34:53.066 DEBUG:teuthology.orchestra.run.smithi022:> sudo /usr/sbin/setenforce permissive 2024-02-10T07:34:53.095 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2024-02-10T07:34:53.151 DEBUG:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2024-02-10T07:34:53.189 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-10T07:34:53.199 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-10T07:34:53.220 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-10T07:34:53.221 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi022.front.sepia.ceph.com,smithi027.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-10T07:39:11.563 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi022.front.sepia.ceph.com'), Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2024-02-10T07:39:11.568 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:39:11.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi022.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:39:11.661 DEBUG:teuthology.orchestra.run.smithi022:> true 2024-02-10T07:39:11.739 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi022.front.sepia.ceph.com' 2024-02-10T07:39:11.740 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:39:11.740 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:39:11.843 DEBUG:teuthology.orchestra.run.smithi027:> true 2024-02-10T07:39:11.928 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2024-02-10T07:39:11.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:39:11.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi081.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-10T07:39:12.021 DEBUG:teuthology.orchestra.run.smithi081:> true 2024-02-10T07:39:12.100 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi081.front.sepia.ceph.com' 2024-02-10T07:39:12.100 INFO:teuthology.run_tasks:Running task clock... 2024-02-10T07:39:12.111 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-10T07:39:12.111 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T07:39:12.111 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:39:12.114 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T07:39:12.114 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:39:12.116 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-10T07:39:12.116 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:39:12.147 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T07:39:12.150 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T07:39:12.165 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T07:39:12.172 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T07:39:12.173 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-02-10T07:39:12.191 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-02-10T07:39:12.197 INFO:teuthology.orchestra.run.smithi022.stderr:sudo: ntpd: command not found 2024-02-10T07:39:12.209 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2024-02-10T07:39:12.211 INFO:teuthology.orchestra.run.smithi022.stdout:506 Cannot talk to daemon 2024-02-10T07:39:12.221 INFO:teuthology.orchestra.run.smithi081.stderr:sudo: ntpd: command not found 2024-02-10T07:39:12.226 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2024-02-10T07:39:12.228 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T07:39:12.235 INFO:teuthology.orchestra.run.smithi081.stdout:506 Cannot talk to daemon 2024-02-10T07:39:12.245 INFO:teuthology.orchestra.run.smithi022.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T07:39:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T07:39:12.252 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-02-10T07:39:12.266 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T07:39:12.268 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-02-10T07:39:12.297 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:39:12.300 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:39:12.301 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-02-10T07:39:12.301 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.301 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.301 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.301 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.324 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:39:12.324 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:39:12.328 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:39:12.328 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-02-10T07:39:12.328 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:39:12.329 INFO:teuthology.run_tasks:Running task pexec... 2024-02-10T07:39:12.339 INFO:teuthology.task.pexec:Executing custom commands... 2024-02-10T07:39:12.339 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi022.front.sepia.ceph.com 2024-02-10T07:39:12.339 DEBUG:teuthology.orchestra.run.smithi022:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T07:39:12.340 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi027.front.sepia.ceph.com 2024-02-10T07:39:12.340 DEBUG:teuthology.orchestra.run.smithi027:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T07:39:12.340 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi081.front.sepia.ceph.com 2024-02-10T07:39:12.340 DEBUG:teuthology.orchestra.run.smithi081:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-02-10T07:39:12.952 INFO:teuthology.orchestra.run.smithi022.stdout:Last metadata expiration check: 0:00:51 ago on Sat 10 Feb 2024 07:38:21 AM UTC. 2024-02-10T07:39:12.970 INFO:teuthology.orchestra.run.smithi081.stdout:Last metadata expiration check: 0:00:55 ago on Sat 10 Feb 2024 07:38:17 AM UTC. 2024-02-10T07:39:13.003 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:48 ago on Sat 10 Feb 2024 07:38:24 AM UTC. 2024-02-10T07:39:13.114 INFO:teuthology.orchestra.run.smithi022.stdout:Dependencies resolved. 2024-02-10T07:39:13.115 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:39:13.115 INFO:teuthology.orchestra.run.smithi022.stdout: Package Architecture Version Repository Size 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Installing: 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction Summary 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:================================================================================ 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Install 1 Package 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Total download size: 3.1 M 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Installed size: 9.8 M 2024-02-10T07:39:13.116 INFO:teuthology.orchestra.run.smithi022.stdout:Downloading Packages: 2024-02-10T07:39:13.125 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies resolved. 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout: Package Architecture Version Repository Size 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T07:39:13.136 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 3.1 M 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 9.8 M 2024-02-10T07:39:13.137 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading Packages: 2024-02-10T07:39:13.160 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout: runc x86_64 4:1.1.12-1.el9 CentOS-AppStream 3.1 M 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2024-02-10T07:39:13.170 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:39:13.171 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.1 M 2024-02-10T07:39:13.171 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 9.8 M 2024-02-10T07:39:13.171 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2024-02-10T07:39:13.439 INFO:teuthology.orchestra.run.smithi022.stdout:runc-1.1.12-1.el9.x86_64.rpm 9.7 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.440 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:39:13.440 INFO:teuthology.orchestra.run.smithi022.stdout:Total 9.6 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.440 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction check 2024-02-10T07:39:13.445 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction check succeeded. 2024-02-10T07:39:13.446 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction test 2024-02-10T07:39:13.455 INFO:teuthology.orchestra.run.smithi081.stdout:runc-1.1.12-1.el9.x86_64.rpm 9.8 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.456 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:39:13.456 INFO:teuthology.orchestra.run.smithi081.stdout:Total 9.8 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.456 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2024-02-10T07:39:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction check succeeded. 2024-02-10T07:39:13.462 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2024-02-10T07:39:13.487 INFO:teuthology.orchestra.run.smithi027.stdout:runc-1.1.12-1.el9.x86_64.rpm 9.9 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.488 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:39:13.488 INFO:teuthology.orchestra.run.smithi027.stdout:Total 9.8 MB/s | 3.1 MB 00:00 2024-02-10T07:39:13.488 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2024-02-10T07:39:13.494 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2024-02-10T07:39:13.494 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2024-02-10T07:39:13.519 INFO:teuthology.orchestra.run.smithi022.stdout:Transaction test succeeded. 2024-02-10T07:39:13.519 INFO:teuthology.orchestra.run.smithi022.stdout:Running transaction 2024-02-10T07:39:13.532 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded. 2024-02-10T07:39:13.532 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2024-02-10T07:39:13.570 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2024-02-10T07:39:13.570 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2024-02-10T07:39:13.719 INFO:teuthology.orchestra.run.smithi022.stdout: Preparing : 1/1 2024-02-10T07:39:13.736 INFO:teuthology.orchestra.run.smithi081.stdout: Preparing : 1/1 2024-02-10T07:39:13.795 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2024-02-10T07:39:13.802 INFO:teuthology.orchestra.run.smithi022.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:13.833 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:13.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.283 INFO:teuthology.orchestra.run.smithi081.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.374 INFO:teuthology.orchestra.run.smithi022.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:39:14.590 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout:Installed: 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:39:14.690 INFO:teuthology.orchestra.run.smithi022.stdout:Complete! 2024-02-10T07:39:14.820 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : runc-4:1.1.12-1.el9.x86_64 1/1 2024-02-10T07:39:14.820 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:39:14.820 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2024-02-10T07:39:14.820 INFO:teuthology.orchestra.run.smithi027.stdout: runc-4:1.1.12-1.el9.x86_64 2024-02-10T07:39:14.820 INFO:teuthology.orchestra.run.smithi027.stdout: 2024-02-10T07:39:14.821 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2024-02-10T07:39:14.832 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:14.913 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:15.073 DEBUG:teuthology.parallel:result is None 2024-02-10T07:39:15.073 INFO:teuthology.run_tasks:Running task nvme_loop... 2024-02-10T07:39:15.084 INFO:tasks.nvme_loop:Setting up nvme_loop on scratch devices... 2024-02-10T07:39:15.084 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:39:15.084 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:15.099 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T07:39:15.099 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_1 2024-02-10T07:39:15.153 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T07:39:15.153 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:15.153 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:39:10.539655954 +0000 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:38:44.334992942 +0000 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:38:44.334992942 +0000 2024-02-10T07:39:15.154 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:38:44.334992942 +0000 2024-02-10T07:39:15.154 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T07:39:15.218 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:39:15.218 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:39:15.218 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 9.1738e-05 s, 5.6 MB/s 2024-02-10T07:39:15.219 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T07:39:15.274 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_2 2024-02-10T07:39:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T07:39:15.328 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:15.328 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-02-10T07:39:15.328 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:15.329 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:15.329 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:39:10.539655954 +0000 2024-02-10T07:39:15.329 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:38:44.680988493 +0000 2024-02-10T07:39:15.329 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:38:44.680988493 +0000 2024-02-10T07:39:15.329 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:38:44.680988493 +0000 2024-02-10T07:39:15.329 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T07:39:15.393 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:39:15.393 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:39:15.393 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 8.5029e-05 s, 6.0 MB/s 2024-02-10T07:39:15.394 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T07:39:15.449 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_3 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:39:10.539655954 +0000 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:38:45.000984378 +0000 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:38:45.000984378 +0000 2024-02-10T07:39:15.505 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:38:45.000984378 +0000 2024-02-10T07:39:15.506 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T07:39:15.570 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:39:15.570 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:39:15.570 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 8.1665e-05 s, 6.3 MB/s 2024-02-10T07:39:15.571 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T07:39:15.626 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/vg_nvme/lv_4 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:39:10.539655954 +0000 2024-02-10T07:39:15.681 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:38:45.337980044 +0000 2024-02-10T07:39:15.682 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:38:45.337980044 +0000 2024-02-10T07:39:15.682 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:38:45.337980044 +0000 2024-02-10T07:39:15.682 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T07:39:15.746 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:39:15.746 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:39:15.746 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 7.9434e-05 s, 6.4 MB/s 2024-02-10T07:39:15.747 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T07:39:15.802 DEBUG:teuthology.orchestra.run.smithi022:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T07:39:15.942 INFO:teuthology.orchestra.run.smithi022.stdout:loop 2024-02-10T07:39:15.944 INFO:tasks.nvme_loop:Connecting nvme_loop smithi022:/dev/vg_nvme/lv_1... 2024-02-10T07:39:15.944 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T07:39:15.981 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-02-10T07:39:16.019 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/vg_nvme/lv_11 2024-02-10T07:39:16.055 INFO:tasks.nvme_loop:Connecting nvme_loop smithi022:/dev/vg_nvme/lv_2... 2024-02-10T07:39:16.055 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T07:39:16.135 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-02-10T07:39:16.174 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/vg_nvme/lv_21 2024-02-10T07:39:16.204 INFO:tasks.nvme_loop:Connecting nvme_loop smithi022:/dev/vg_nvme/lv_3... 2024-02-10T07:39:16.204 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T07:39:16.283 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-02-10T07:39:16.322 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/vg_nvme/lv_31 2024-02-10T07:39:16.360 INFO:tasks.nvme_loop:Connecting nvme_loop smithi022:/dev/vg_nvme/lv_4... 2024-02-10T07:39:16.360 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T07:39:16.439 INFO:teuthology.orchestra.run.smithi022.stdout:1 2024-02-10T07:39:16.479 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/vg_nvme/lv_41 2024-02-10T07:39:16.517 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:39:16.517 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:16.572 DEBUG:teuthology.orchestra.run.smithi022:> sudo nvme list 2024-02-10T07:39:16.638 INFO:teuthology.orchestra.run.smithi022.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T07:39:16.638 INFO:teuthology.orchestra.run.smithi022.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T07:39:16.638 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/nvme0n1 PHMB747402EC480DGN INTEL SSDPED1D480GA 1 480.10 GB / 480.10 GB 512 B + 0 B E2010325 2024-02-10T07:39:16.638 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/nvme1n1 195905c835d344ec2706 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:16.638 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/nvme2n1 8598b221f58a6e76262d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:16.639 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/nvme3n1 f7bab0afee587262db69 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:16.639 INFO:teuthology.orchestra.run.smithi022.stdout:/dev/nvme4n1 4c6f184022173ed3ece2 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:16.639 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:39:16.640 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:39:16.640 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/scratch_devs 2024-02-10T07:39:16.707 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:39:16.708 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:16.727 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T07:39:16.727 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2024-02-10T07:39:16.787 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T07:39:16.787 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:16.787 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-02-10T07:39:16.787 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:16.788 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:16.788 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:39:10.500903344 +0000 2024-02-10T07:39:16.788 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:38:48.737477299 +0000 2024-02-10T07:39:16.788 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:38:48.737477299 +0000 2024-02-10T07:39:16.788 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:38:48.737477299 +0000 2024-02-10T07:39:16.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T07:39:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:39:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:39:16.862 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000221009 s, 2.3 MB/s 2024-02-10T07:39:16.864 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T07:39:16.923 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2024-02-10T07:39:16.980 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T07:39:16.980 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-02-10T07:39:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:39:10.500903344 +0000 2024-02-10T07:39:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:38:49.094467880 +0000 2024-02-10T07:39:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:38:49.094467880 +0000 2024-02-10T07:39:16.981 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:38:49.094467880 +0000 2024-02-10T07:39:16.981 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T07:39:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:39:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:39:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000196773 s, 2.6 MB/s 2024-02-10T07:39:17.053 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T07:39:17.110 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:39:10.500903344 +0000 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:38:49.432458962 +0000 2024-02-10T07:39:17.166 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:38:49.432458962 +0000 2024-02-10T07:39:17.167 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:38:49.432458962 +0000 2024-02-10T07:39:17.167 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T07:39:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:39:17.236 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:39:17.237 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000274249 s, 1.9 MB/s 2024-02-10T07:39:17.238 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T07:39:17.294 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2024-02-10T07:39:17.350 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T07:39:17.350 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-02-10T07:39:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:17.350 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:39:10.500903344 +0000 2024-02-10T07:39:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:38:49.784449675 +0000 2024-02-10T07:39:17.351 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:38:49.784449675 +0000 2024-02-10T07:39:17.351 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:38:49.784449675 +0000 2024-02-10T07:39:17.351 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T07:39:17.419 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:39:17.419 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:39:17.420 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000184255 s, 2.8 MB/s 2024-02-10T07:39:17.421 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T07:39:17.481 DEBUG:teuthology.orchestra.run.smithi027:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T07:39:17.640 INFO:teuthology.orchestra.run.smithi027.stdout:loop 2024-02-10T07:39:17.641 INFO:tasks.nvme_loop:Connecting nvme_loop smithi027:/dev/vg_nvme/lv_1... 2024-02-10T07:39:17.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T07:39:17.694 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-02-10T07:39:17.742 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/vg_nvme/lv_11 2024-02-10T07:39:17.778 INFO:tasks.nvme_loop:Connecting nvme_loop smithi027:/dev/vg_nvme/lv_2... 2024-02-10T07:39:17.778 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T07:39:17.871 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-02-10T07:39:17.922 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/vg_nvme/lv_21 2024-02-10T07:39:17.958 INFO:tasks.nvme_loop:Connecting nvme_loop smithi027:/dev/vg_nvme/lv_3... 2024-02-10T07:39:17.958 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T07:39:18.045 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-02-10T07:39:18.095 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/vg_nvme/lv_31 2024-02-10T07:39:18.131 INFO:tasks.nvme_loop:Connecting nvme_loop smithi027:/dev/vg_nvme/lv_4... 2024-02-10T07:39:18.132 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T07:39:18.181 INFO:teuthology.orchestra.run.smithi027.stdout:1 2024-02-10T07:39:18.232 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/vg_nvme/lv_41 2024-02-10T07:39:18.276 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:39:18.276 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:18.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo nvme list 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/nvme0n1 CVFT5331001C400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/nvme1n1 d6b182bfade4c62e29c5 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/nvme2n1 c214e663dd948f2ad906 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/nvme3n1 d7d2419c00bb0d29e2d9 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:/dev/nvme4n1 2cf39b667b9184791ebb Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:18.415 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:39:18.416 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:39:18.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/scratch_devs 2024-02-10T07:39:18.487 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:39:18.487 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:18.507 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-10T07:39:18.507 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 2024-02-10T07:39:18.565 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-10T07:39:18.565 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:18.565 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-02-10T07:39:18.565 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:18.565 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:18.566 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:39:10.570304072 +0000 2024-02-10T07:39:18.566 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:38:41.897852702 +0000 2024-02-10T07:39:18.566 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:38:41.897852702 +0000 2024-02-10T07:39:18.566 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:38:41.897852702 +0000 2024-02-10T07:39:18.566 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-10T07:39:18.636 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:39:18.636 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:39:18.636 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.00023801 s, 2.2 MB/s 2024-02-10T07:39:18.638 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-10T07:39:18.696 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 2024-02-10T07:39:18.753 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-10T07:39:18.753 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:39:10.570304072 +0000 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:38:42.245846033 +0000 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:38:42.245846033 +0000 2024-02-10T07:39:18.754 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:38:42.245846033 +0000 2024-02-10T07:39:18.754 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-10T07:39:18.822 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:39:18.822 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:39:18.822 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000206738 s, 2.5 MB/s 2024-02-10T07:39:18.823 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-10T07:39:18.881 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 2024-02-10T07:39:18.938 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:39:10.570304072 +0000 2024-02-10T07:39:18.939 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:38:42.595839326 +0000 2024-02-10T07:39:18.940 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:38:42.595839326 +0000 2024-02-10T07:39:18.940 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:38:42.595839326 +0000 2024-02-10T07:39:18.940 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-10T07:39:19.007 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:39:19.007 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:39:19.007 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000214146 s, 2.4 MB/s 2024-02-10T07:39:19.009 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-10T07:39:19.067 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:39:10.570304072 +0000 2024-02-10T07:39:19.123 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:38:42.933832848 +0000 2024-02-10T07:39:19.124 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:38:42.933832848 +0000 2024-02-10T07:39:19.124 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:38:42.933832848 +0000 2024-02-10T07:39:19.124 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-10T07:39:19.190 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:39:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:39:19.191 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000214039 s, 2.4 MB/s 2024-02-10T07:39:19.192 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-10T07:39:19.249 DEBUG:teuthology.orchestra.run.smithi081:> grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype 2024-02-10T07:39:19.389 INFO:teuthology.orchestra.run.smithi081.stdout:loop 2024-02-10T07:39:19.391 INFO:tasks.nvme_loop:Connecting nvme_loop smithi081:/dev/vg_nvme/lv_1... 2024-02-10T07:39:19.391 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1 && echo -n /dev/vg_nvme/lv_1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_1/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_1 /sys/kernel/config/nvmet/ports/1/subsystems/lv_1 && sudo nvme connect -t loop -n lv_1 -q hostnqn 2024-02-10T07:39:19.428 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-02-10T07:39:19.465 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/vg_nvme/lv_11 2024-02-10T07:39:19.494 INFO:tasks.nvme_loop:Connecting nvme_loop smithi081:/dev/vg_nvme/lv_2... 2024-02-10T07:39:19.494 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1 && echo -n /dev/vg_nvme/lv_2 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_2/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_2 /sys/kernel/config/nvmet/ports/1/subsystems/lv_2 && sudo nvme connect -t loop -n lv_2 -q hostnqn 2024-02-10T07:39:19.573 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-02-10T07:39:19.616 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/vg_nvme/lv_21 2024-02-10T07:39:19.646 INFO:tasks.nvme_loop:Connecting nvme_loop smithi081:/dev/vg_nvme/lv_3... 2024-02-10T07:39:19.646 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1 && echo -n /dev/vg_nvme/lv_3 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_3/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_3 /sys/kernel/config/nvmet/ports/1/subsystems/lv_3 && sudo nvme connect -t loop -n lv_3 -q hostnqn 2024-02-10T07:39:19.724 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-02-10T07:39:19.762 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/vg_nvme/lv_31 2024-02-10T07:39:19.791 INFO:tasks.nvme_loop:Connecting nvme_loop smithi081:/dev/vg_nvme/lv_4... 2024-02-10T07:39:19.791 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4 && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/attr_allow_any_host && sudo mkdir -p /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1 && echo -n /dev/vg_nvme/lv_4 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/device_path && echo 1 | sudo tee /sys/kernel/config/nvmet/subsystems/lv_4/namespaces/1/enable && sudo ln -s /sys/kernel/config/nvmet/subsystems/lv_4 /sys/kernel/config/nvmet/ports/1/subsystems/lv_4 && sudo nvme connect -t loop -n lv_4 -q hostnqn 2024-02-10T07:39:19.870 INFO:teuthology.orchestra.run.smithi081.stdout:1 2024-02-10T07:39:19.908 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/vg_nvme/lv_41 2024-02-10T07:39:19.945 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:39:19.945 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:39:20.001 DEBUG:teuthology.orchestra.run.smithi081:> sudo nvme list 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:Node SN Model Namespace Usage Format FW Rev 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:---------------- -------------------- ---------------------------------------- --------- -------------------------- ---------------- -------- 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/nvme0n1 PHFT620400N7400BGN INTEL SSDPEDMD400G4 1 400.09 GB / 400.09 GB 512 B + 0 B 8DV101H0 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/nvme1n1 a3a5c538dee4d86a70f3 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/nvme2n1 d073ecffb9d6e1fc926d Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/nvme3n1 58a182ce31680661db17 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:20.067 INFO:teuthology.orchestra.run.smithi081.stdout:/dev/nvme4n1 ff3093766c18c2b1ff54 Linux 1 96.00 GB / 96.00 GB 512 B + 0 B 5.14.0-4 2024-02-10T07:39:20.068 INFO:tasks.nvme_loop:new_devs ['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:39:20.069 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:39:20.069 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/scratch_devs 2024-02-10T07:39:20.135 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-10T07:39:20.239 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1, 'mgr/cephadm/use_agent': False}, 'global': {'mon election default strategy': 3}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90'} 2024-02-10T07:39:20.239 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:39:20.240 INFO:tasks.cephadm:Cluster fsid is 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:39:20.240 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-10T07:39:20.240 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.22', 'mon.b': '172.21.15.27', 'mon.c': '172.21.15.81'} 2024-02-10T07:39:20.240 INFO:tasks.cephadm:First mon is mon.a on smithi022 2024-02-10T07:39:20.240 INFO:tasks.cephadm:First mgr is y 2024-02-10T07:39:20.240 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-10T07:39:20.240 DEBUG:teuthology.orchestra.run.smithi022:> sudo hostname $(hostname -s) 2024-02-10T07:39:20.273 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2024-02-10T07:39:20.310 DEBUG:teuthology.orchestra.run.smithi081:> sudo hostname $(hostname -s) 2024-02-10T07:39:20.337 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-02-10T07:39:20.338 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:39:20.502 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/76638/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-1361-g290212a1', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-1361.g290212a1'}, 'url': 'https://2.chacra.ceph.com/r/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-02-09 18:34:21.832169', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'main', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-02-10T07:39:20.679 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref main, sha1 290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:39:20.681 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm 2024-02-10T07:39:20.681 DEBUG:teuthology.orchestra.run.smithi022:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.111 INFO:teuthology.orchestra.run.smithi022.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 07:39 /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.111 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.566 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 07:39 /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.567 DEBUG:teuthology.orchestra.run.smithi081:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/main/290212a1bfd33563ba76ee5b7f0b944fdb6c5d90/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.987 INFO:teuthology.orchestra.run.smithi081.stdout:-rw-r--r--. 1 ubuntu ubuntu 766809 Feb 10 07:39 /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:21.988 DEBUG:teuthology.orchestra.run.smithi022:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:22.008 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:22.028 DEBUG:teuthology.orchestra.run.smithi081:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-02-10T07:39:22.059 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 on all hosts... 2024-02-10T07:39:22.059 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T07:39:22.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T07:39:22.071 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 pull 2024-02-10T07:39:22.843 INFO:teuthology.orchestra.run.smithi022.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:39:22.942 INFO:teuthology.orchestra.run.smithi081.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:39:22.966 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:40:08.356 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "repo_digests": [ 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2024-02-10T07:40:08.357 INFO:teuthology.orchestra.run.smithi027.stdout:} 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout:{ 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout: "repo_digests": [ 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout: ] 2024-02-10T07:40:11.148 INFO:teuthology.orchestra.run.smithi022.stdout:} 2024-02-10T07:40:11.261 INFO:teuthology.orchestra.run.smithi081.stdout:{ 2024-02-10T07:40:11.261 INFO:teuthology.orchestra.run.smithi081.stdout: "ceph_version": "ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev)", 2024-02-10T07:40:11.261 INFO:teuthology.orchestra.run.smithi081.stdout: "image_id": "c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6", 2024-02-10T07:40:11.261 INFO:teuthology.orchestra.run.smithi081.stdout: "repo_digests": [ 2024-02-10T07:40:11.262 INFO:teuthology.orchestra.run.smithi081.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58" 2024-02-10T07:40:11.262 INFO:teuthology.orchestra.run.smithi081.stdout: ] 2024-02-10T07:40:11.262 INFO:teuthology.orchestra.run.smithi081.stdout:} 2024-02-10T07:40:11.285 DEBUG:teuthology.orchestra.run.smithi022:> sudo mkdir -p /etc/ceph 2024-02-10T07:40:11.315 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2024-02-10T07:40:11.355 DEBUG:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph 2024-02-10T07:40:11.384 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 777 /etc/ceph 2024-02-10T07:40:11.411 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2024-02-10T07:40:11.446 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 777 /etc/ceph 2024-02-10T07:40:11.472 INFO:tasks.cephadm:Writing seed config... 2024-02-10T07:40:11.473 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-10T07:40:11.473 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-10T07:40:11.473 INFO:tasks.cephadm: override: [mgr] mgr/cephadm/use_agent = False 2024-02-10T07:40:11.473 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-10T07:40:11.474 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-10T07:40:11.474 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:40:11.474 DEBUG:teuthology.orchestra.run.smithi022:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-10T07:40:11.489 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 80815fa4-c7e7-11ee-95b8-87774f69a715 mon election default strategy = 3 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 mgr/cephadm/use_agent = False [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-02-10T07:40:11.489 DEBUG:teuthology.orchestra.run.smithi022:mon.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service 2024-02-10T07:40:11.531 DEBUG:teuthology.orchestra.run.smithi022:mgr.y> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service 2024-02-10T07:40:11.574 INFO:tasks.cephadm:Bootstrapping... 2024-02-10T07:40:11.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 -v bootstrap --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.22 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:40:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:-------------------------------------------------------------------------------- 2024-02-10T07:40:11.768 INFO:teuthology.orchestra.run.smithi022.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90', '-v', 'bootstrap', '--fsid', '80815fa4-c7e7-11ee-95b8-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.22', '--skip-admin-label'] 2024-02-10T07:40:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:40:11.793 INFO:teuthology.orchestra.run.smithi022.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-02-10T07:40:11.793 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying podman|docker is present... 2024-02-10T07:40:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:40:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying lvm2 is present... 2024-02-10T07:40:11.817 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying time synchronization is in place... 2024-02-10T07:40:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T07:40:11.824 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T07:40:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T07:40:11.830 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:40:11.836 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-02-10T07:40:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-02-10T07:40:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-02-10T07:40:11.842 INFO:teuthology.orchestra.run.smithi022.stdout:Repeating the final host check... 2024-02-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout 4.9.3-dev 2024-02-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi022.stdout:podman (/bin/podman) version 4.9.3 is present 2024-02-10T07:40:11.864 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl is present 2024-02-10T07:40:11.865 INFO:teuthology.orchestra.run.smithi022.stdout:lvcreate is present 2024-02-10T07:40:11.871 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-02-10T07:40:11.871 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-02-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-02-10T07:40:11.876 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:40:11.882 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout enabled 2024-02-10T07:40:11.888 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout active 2024-02-10T07:40:11.888 INFO:teuthology.orchestra.run.smithi022.stdout:Unit chronyd.service is enabled and running 2024-02-10T07:40:11.888 INFO:teuthology.orchestra.run.smithi022.stdout:Host looks OK 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Cluster fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Acquiring lock 140542960155232 on /run/cephadm/80815fa4-c7e7-11ee-95b8-87774f69a715.lock 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Lock 140542960155232 acquired on /run/cephadm/80815fa4-c7e7-11ee-95b8-87774f69a715.lock 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 3300 ... 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying IP 172.21.15.22 port 6789 ... 2024-02-10T07:40:11.889 INFO:teuthology.orchestra.run.smithi022.stdout:Base mon IP(s) is [172.21.15.22:3300, 172.21.15.22:6789], mon addrv is [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-02-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.22 metric 100 2024-02-10T07:40:11.892 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.22 metric 100 2024-02-10T07:40:11.895 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-02-10T07:40:11.895 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-02-10T07:40:11.895 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-02-10T07:40:11.897 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-10T07:40:11.897 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-02-10T07:40:11.897 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T07:40:11.897 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-02-10T07:40:11.897 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe59:5de4/64 scope link noprefixroute 2024-02-10T07:40:11.898 INFO:teuthology.orchestra.run.smithi022.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-02-10T07:40:11.898 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-02-10T07:40:11.898 INFO:teuthology.orchestra.run.smithi022.stdout:Mon IP `172.21.15.22` is in CIDR network `172.21.0.0/20` 2024-02-10T07:40:11.898 INFO:teuthology.orchestra.run.smithi022.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-02-10T07:40:11.898 INFO:teuthology.orchestra.run.smithi022.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-10T07:40:11.899 INFO:teuthology.orchestra.run.smithi022.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:40:12.384 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stdout c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90... 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Getting image source signatures 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:58df4e03ba066accf3a7e07426bab5f1c867e27a139801c642bdafb42b9a2d9a 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying blob sha256:5fe0e97a267976e17c7dcb240cf9455472de6876101ae78af4bdf182158b6d8d 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Copying config sha256:c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T07:40:12.385 INFO:teuthology.orchestra.run.smithi022.stdout:/bin/podman: stderr Writing manifest to image destination 2024-02-10T07:40:13.323 INFO:teuthology.orchestra.run.smithi022.stdout:ceph: stdout ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T07:40:13.323 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph version: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev) 2024-02-10T07:40:13.323 INFO:teuthology.orchestra.run.smithi022.stdout:Extracting ceph user uid/gid from container image... 2024-02-10T07:40:14.160 INFO:teuthology.orchestra.run.smithi022.stdout:stat: stdout 167 167 2024-02-10T07:40:14.161 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial keys... 2024-02-10T07:40:15.283 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQBeKMdl+bWlJxAA+JI5xVweUAleDuHVgGWI0w== 2024-02-10T07:40:16.346 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQBfKMdle24JJxAAV10JJK5/jk0HE/42l1dGuA== 2024-02-10T07:40:17.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph-authtool: stdout AQBgKMdlne46JxAAnBmkIk3t9m8sHRNyEt6Q7A== 2024-02-10T07:40:17.169 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial monmap... 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:monmaptool for a [v2:172.21.15.22:3300,v1:172.21.15.22:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:setting min_mon_release = quincy 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: set fsid to 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:18.233 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-10T07:40:18.234 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:40:18.234 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mon... 2024-02-10T07:40:27.450 INFO:teuthology.orchestra.run.smithi022.stdout:create mon.a on 2024-02-10T07:40:27.815 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-02-10T07:40:28.067 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target. 2024-02-10T07:40:28.068 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target → /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target. 2024-02-10T07:40:28.317 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a 2024-02-10T07:40:28.318 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service: Unit ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service not loaded. 2024-02-10T07:40:28.483 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target.wants/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service → /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@.service. 2024-02-10T07:40:28.992 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:28 smithi022 ceph-mon[25216]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T07:40:29.005 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:40:29.005 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-10T07:40:29.011 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:40:29.011 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:40:29.011 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-10T07:40:29.012 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T07:40:29.012 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon to start... 2024-02-10T07:40:29.012 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mon... 2024-02-10T07:40:29.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:28 smithi022 bash[25177]: 9ad13c1efca820b87d5a140094f6a4bf05ea4545f7e91171fb5008a3e8b14147 2024-02-10T07:40:29.253 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:28 smithi022 systemd[1]: Started Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout cluster: 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout id: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout services: 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.593945s) 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-02-10T07:40:30.028 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout data: 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout pgs: 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:mon is available 2024-02-10T07:40:30.029 INFO:teuthology.orchestra.run.smithi022.stdout:Assimilating anything we can from ceph.conf... 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: fsmap 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: mgrmap e1: no daemons active 2024-02-10T07:40:30.141 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:29 smithi022 ceph-mon[25216]: from='client.? 172.21.15.22:0/2012122079' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-10T07:40:31.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:30 smithi022 ceph-mon[25216]: from='client.? 172.21.15.22:0/2693273588' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T07:40:31.155 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:30 smithi022 ceph-mon[25216]: from='client.? 172.21.15.22:0/2693273588' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.22:3300,v1:172.21.15.22:6789] 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T07:40:31.159 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/cephadm/use_agent = False 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T07:40:31.160 INFO:teuthology.orchestra.run.smithi022.stdout:Generating new minimal ceph.conf... 2024-02-10T07:40:32.027 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:31 smithi022 ceph-mon[25216]: from='client.? 172.21.15.22:0/3651003679' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:40:32.313 INFO:teuthology.orchestra.run.smithi022.stdout:Restarting the monitor... 2024-02-10T07:40:32.634 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:32 smithi022 systemd[1]: Stopping Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:40:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a[25193]: 2024-02-10T07:40:32.633+0000 7fc761567700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:40:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a[25193]: 2024-02-10T07:40:32.633+0000 7fc761567700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-10T07:40:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:32 smithi022 podman[25543]: 2024-02-10 07:40:32.705310908 +0000 UTC m=+0.229118636 container died 9ad13c1efca820b87d5a140094f6a4bf05ea4545f7e91171fb5008a3e8b14147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:40:33.846 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:33 smithi022 podman[25543]: 2024-02-10 07:40:33.570731291 +0000 UTC m=+1.094539004 container cleanup 9ad13c1efca820b87d5a140094f6a4bf05ea4545f7e91171fb5008a3e8b14147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=main-290212a, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=) 2024-02-10T07:40:33.847 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:33 smithi022 bash[25543]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a 2024-02-10T07:40:34.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:33 smithi022 podman[25562]: 2024-02-10 07:40:33.846465934 +0000 UTC m=+1.138488030 container remove 9ad13c1efca820b87d5a140094f6a4bf05ea4545f7e91171fb5008a3e8b14147 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, RELEASE=main-290212a, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-02-10T07:40:34.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:33 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service: Deactivated successfully. 2024-02-10T07:40:34.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 systemd[1]: Stopped Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:40:34.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 systemd[1]: Starting Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 podman[25666]: 2024-02-10 07:40:34.113784341 +0000 UTC m=+0.017818007 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 podman[25666]: 2024-02-10 07:40:34.214078015 +0000 UTC m=+0.118111702 container create dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-02-10T07:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 podman[25666]: 2024-02-10 07:40:34.40803117 +0000 UTC m=+0.312064825 container init dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2) 2024-02-10T07:40:34.499 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 podman[25666]: 2024-02-10 07:40:34.413649785 +0000 UTC m=+0.317683428 container start dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-290212a, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: pidfile_write: ignore empty --pid-file 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: load: jerasure load: lrc 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: RocksDB version: 7.9.2 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Git sha 0 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: DB SUMMARY 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: DB Session ID: PLSQFT4WBBWZ1TQ0Z9QB 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: CURRENT file: CURRENT 2024-02-10T07:40:34.500 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T07:40:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-02-10T07:40:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-02-10T07:40:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 84627 ; 2024-02-10T07:40:34.501 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.error_if_exists: 0 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.create_if_missing: 0 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.paranoid_checks: 1 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.env: 0x5646a59e7ae0 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.info_log: 0x5646a79818c0 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T07:40:34.502 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.statistics: (nil) 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.use_fsync: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_log_file_size: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_fallocate: 1 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.use_direct_reads: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T07:40:34.503 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.db_log_dir: 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.wal_dir: 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T07:40:34.504 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.write_buffer_manager: 0x5646a7955400 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T07:40:34.505 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.unordered_write: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.row_cache: None 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.wal_filter: None 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.two_write_queues: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.wal_compression: 0 2024-02-10T07:40:34.506 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.atomic_flush: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.log_readahead_size: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_background_jobs: 2 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_background_compactions: -1 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_subcompactions: 1 2024-02-10T07:40:34.507 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_open_files: -1 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T07:40:34.508 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_background_flushes: -1 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Compression algorithms supported: 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kZSTD supported: 0 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kXpressCompression supported: 0 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kZlibCompression supported: 1 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kSnappyCompression supported: 1 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kLZ4Compression supported: 1 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: kBZip2Compression supported: 0 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T07:40:34.509 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.merge_operator: 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_filter: None 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5646a79819c0) 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks: 1 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T07:40:34.510 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: pin_top_level_index_and_filter: 1 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_type: 0 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_index_type: 0 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_shortening: 1 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: checksum: 4 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: no_block_cache: 0 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache: 0x5646a7985090 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_name: BinnedLRUCache 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_options: 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: capacity : 536870912 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_shard_bits : 4 2024-02-10T07:40:34.511 INFO:journalctl@ceph.mon.a.smithi022.stdout: strict_capacity_limit : 0 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: high_pri_pool_ratio: 0.000 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_cache_compressed: (nil) 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: persistent_cache: (nil) 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size: 4096 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_size_deviation: 10 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_restart_interval: 16 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: index_block_restart_interval: 1 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: metadata_block_size: 4096 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: partition_filters: 0 2024-02-10T07:40:34.512 INFO:journalctl@ceph.mon.a.smithi022.stdout: use_delta_encoding: 1 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: filter_policy: bloomfilter 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: whole_key_filtering: 1 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: verify_compression: 0 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: read_amp_bytes_per_bit: 0 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: format_version: 5 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_index_compression: 1 2024-02-10T07:40:34.513 INFO:journalctl@ceph.mon.a.smithi022.stdout: block_align: 0 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout: max_auto_readahead_size: 262144 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout: prepopulate_block_cache: 0 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout: initial_auto_readahead_size: 8192 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression: NoCompression 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T07:40:34.514 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.num_levels: 7 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:40:34.515 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:40:34.516 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T07:40:34.517 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T07:40:34.517 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T07:40:34.517 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T07:40:34.518 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T07:40:34.519 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T07:40:34.520 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.table_properties_collectors: 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.inplace_update_support: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.bloom_locality: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.max_successive_merges: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T07:40:34.521 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.ttl: 2592000 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enable_blob_files: false 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.min_blob_size: 0 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T07:40:34.522 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 707abd42-2075-4c1d-b616-0b49792c825e 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550834442275, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550834443113, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 81252, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 252, "table_properties": {"data_size": 79439, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10027, "raw_average_key_size": 47, "raw_value_size": 73748, "raw_average_value_size": 349, "num_data_blocks": 10, "num_entries": 211, "num_filter_entries": 211, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707550834, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "707abd42-2075-4c1d-b616-0b49792c825e", "db_session_id": "PLSQFT4WBBWZ1TQ0Z9QB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550834443195, "job": 1, "event": "recovery_finished"} 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-02-10T07:40:34.523 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5646a7a76000 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: DB pointer 0x5646a7a60000 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** DB Stats ** 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:40:34.524 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: L0 2/0 81.18 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: Sum 2/0 81.18 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** Compaction Stats [default] ** 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T07:40:34.525 INFO:journalctl@ceph.mon.a.smithi022.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.8 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Cumulative compaction: 0.00 GB write, 27.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Interval compaction: 0.00 GB write, 27.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:40:34.526 INFO:journalctl@ceph.mon.a.smithi022.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache BinnedLRUCache@0x5646a7985090#7 capacity: 512.00 MB usage: 1.17 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.41 KB,7.7486e-05%) Misc(1,0.00 KB,0%) 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: starting mon.a rank 0 at public addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] at bind addrs [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???) e1 preinit fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).mds e1 new map 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).mds e1 print_map 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: e1 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: legacy client fscid: -1 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout: No filesystems configured 2024-02-10T07:40:34.527 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 bash[25666]: dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 2024-02-10T07:40:34.528 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 systemd[1]: Started Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:40:34.528 INFO:teuthology.orchestra.run.smithi022.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-02-10T07:40:34.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mon.a is new leader, mons a in quorum (ranks 0) 2024-02-10T07:40:34.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: monmap e1: 1 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:40:34.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: fsmap 2024-02-10T07:40:34.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: osdmap e1: 0 total, 0 up, 0 in 2024-02-10T07:40:34.844 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:34 smithi022 ceph-mon[25704]: mgrmap e1: no daemons active 2024-02-10T07:40:35.567 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote config to /etc/ceph/ceph.conf 2024-02-10T07:40:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:40:35.588 INFO:teuthology.orchestra.run.smithi022.stdout:Creating mgr... 2024-02-10T07:40:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:9283 ... 2024-02-10T07:40:35.591 INFO:teuthology.orchestra.run.smithi022.stdout:Verifying port 0.0.0.0:8765 ... 2024-02-10T07:40:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y 2024-02-10T07:40:35.839 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Failed to reset failed state of unit ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service: Unit ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service not loaded. 2024-02-10T07:40:35.991 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715.target.wants/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service → /etc/systemd/system/ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@.service. 2024-02-10T07:40:36.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:36 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4213969321' entity='client.admin' 2024-02-10T07:40:36.309 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 podman[25946]: 2024-02-10 07:40:36.12603184 +0000 UTC m=+0.017434192 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 2024-02-10T07:40:36.530 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:40:36.530 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-10T07:40:36.538 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:40:36.538 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:40:36.538 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-10T07:40:36.538 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to enable service . firewalld.service is not available 2024-02-10T07:40:36.546 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:40:36.546 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-10T07:40:36.553 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:40:36.553 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:40:36.553 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-10T07:40:36.554 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-02-10T07:40:36.554 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr to start... 2024-02-10T07:40:36.554 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr... 2024-02-10T07:40:36.677 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 podman[25946]: 2024-02-10 07:40:36.40125428 +0000 UTC m=+0.292656634 container init cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:40:36.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 podman[25946]: 2024-02-10 07:40:36.406672665 +0000 UTC m=+0.298075004 container start cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:40:36.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 bash[25946]: cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 2024-02-10T07:40:36.678 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 systemd[1]: Started Ceph mgr.y for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:40:37.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:37 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3416224070' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:40:37.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:36.968+0000 7fba96ebb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:40:37.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:37 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:37.065+0000 7fba96ebb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "80815fa4-c7e7-11ee-95b8-87774f69a715", 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:40:37.777 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:37.783 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:40:37.784 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.785 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:37.786 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:40:28.952268+0000", 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:37.787 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (1/15)... 2024-02-10T07:40:38.075 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:37 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:37.738+0000 7fba96ebb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:40:38.075 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:37 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:37.834+0000 7fba96ebb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:40:38.075 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:37 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:37.977+0000 7fba96ebb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:40:38.075 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.074+0000 7fba96ebb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:40:38.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.158+0000 7fba96ebb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:40:38.425 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.424+0000 7fba96ebb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:40:38.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.536+0000 7fba96ebb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:40:38.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.623+0000 7fba96ebb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:40:39.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.720+0000 7fba96ebb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:40:39.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:38.897+0000 7fba96ebb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:40:40.151 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:39 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:39.904+0000 7fba96ebb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:40:40.151 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:40.041+0000 7fba96ebb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:40:40.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:40.384+0000 7fba96ebb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:40:40.862 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2679779631' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:40:40.863 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:40.475+0000 7fba96ebb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:40:40.863 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:40.674+0000 7fba96ebb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:40:40.863 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:40 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:40.862+0000 7fba96ebb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:40:40.963 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:40.963 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:40.963 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "80815fa4-c7e7-11ee-95b8-87774f69a715", 2024-02-10T07:40:40.963 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:40:40.963 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:40.964 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:40:40.965 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:40:40.966 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:40:28.952268+0000", 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:40.967 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (2/15)... 2024-02-10T07:40:41.413 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:41 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:41.151+0000 7fba96ebb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:40:41.413 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:41 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:41.412+0000 7fba96ebb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:40:41.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:41 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:41.651+0000 7fba96ebb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:40:42.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:42 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:42.471+0000 7fba96ebb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:40:43.382 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:43.057+0000 7fba96ebb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:40:43.382 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:43.158+0000 7fba96ebb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:40:43.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:43.384+0000 7fba96ebb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:40:43.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: Activating manager daemon y 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: mgrmap e2: y(active, starting, since 0.00306253s) 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: Manager daemon y is now available 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' 2024-02-10T07:40:43.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:43 smithi022 ceph-mon[25704]: from='mgr.14100 172.21.15.22:0/2408133339' entity='mgr.y' 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "80815fa4-c7e7-11ee-95b8-87774f69a715", 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:40:44.016 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:40:44.017 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:40:44.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:40:44.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.021 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:40:44.022 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": false, 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:40:28.952268+0000", 2024-02-10T07:40:44.023 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:44.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:44.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:40:44.024 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:44.024 INFO:teuthology.orchestra.run.smithi022.stdout:mgr not available, waiting (3/15)... 2024-02-10T07:40:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:44 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/588984941' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:40:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:44 smithi022 ceph-mon[25704]: mgrmap e3: y(active, since 1.00723s) 2024-02-10T07:40:46.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:46 smithi022 ceph-mon[25704]: mgrmap e4: y(active, since 2s) 2024-02-10T07:40:47.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:47.351 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsid": "80815fa4-c7e7-11ee-95b8-87774f69a715", 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "health": { 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 0 2024-02-10T07:40:47.352 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "a" 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "monmap": { 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-02-10T07:40:47.353 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.354 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modules": [ 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "iostat", 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "nfs", 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "restful" 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ], 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.355 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "modified": "2024-02-10T07:40:28.952268+0000", 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "services": {} 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout }, 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:47.356 INFO:teuthology.orchestra.run.smithi022.stdout:mgr is available 2024-02-10T07:40:47.770 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:47 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2812425491' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-10T07:40:48.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:48 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1648663874' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-10T07:40:48.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:48 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1648663874' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-10T07:40:48.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:48.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [global] 2024-02-10T07:40:48.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout fsid = 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:40:48.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-02-10T07:40:48.584 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [mgr] 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout [osd] 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-02-10T07:40:48.585 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling cephadm module... 2024-02-10T07:40:49.763 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:49 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/123665601' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-10T07:40:49.763 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:49 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setuser ceph since I am not root 2024-02-10T07:40:49.763 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:49 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setgroup ceph since I am not root 2024-02-10T07:40:50.284 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:50.033+0000 7fcbf47bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:40:50.284 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:50.132+0000 7fcbf47bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:40:50.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/123665601' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-10T07:40:50.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-mon[25704]: mgrmap e5: y(active, since 6s) 2024-02-10T07:40:51.045 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:50.805+0000 7fcbf47bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:40:51.045 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:50 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:50.901+0000 7fcbf47bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:40:51.046 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.045+0000 7fcbf47bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:40:51.336 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.143+0000 7fcbf47bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:40:51.336 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.228+0000 7fcbf47bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-02-10T07:40:51.340 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 5... 2024-02-10T07:40:51.623 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3367686360' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T07:40:51.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.505+0000 7fcbf47bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:40:51.623 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.622+0000 7fcbf47bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:40:51.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.710+0000 7fcbf47bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:40:51.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.810+0000 7fcbf47bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:40:52.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:51 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:51.989+0000 7fcbf47bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:40:53.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:52 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:52.974+0000 7fcbf47bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:40:53.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:53 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:53.108+0000 7fcbf47bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:40:53.735 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:53 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:53.444+0000 7fcbf47bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:40:53.736 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:53 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:53.537+0000 7fcbf47bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:40:53.736 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:53 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:53.735+0000 7fcbf47bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:40:54.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:53 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:53.922+0000 7fcbf47bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:40:54.472 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:54 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:54.211+0000 7fcbf47bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:40:54.472 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:54 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:54.472+0000 7fcbf47bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:40:54.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:54 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:54.712+0000 7fcbf47bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:40:55.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:55 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:55.534+0000 7fcbf47bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:40:56.437 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:56.116+0000 7fcbf47bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:40:56.437 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:56.217+0000 7fcbf47bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:40:56.437 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:40:56.436+0000 7fcbf47bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: Active manager daemon y restarted 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: Activating manager daemon y 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: osdmap e2: 0 total, 0 up, 0 in 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: mgrmap e6: y(active, starting, since 0.00320724s) 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:40:56.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: Manager daemon y is now available 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:56.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:56 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: Found migration_current of "None". Setting to last migration. 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:40:57.767 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:57 smithi022 ceph-mon[25704]: mgrmap e7: y(active, since 1.00493s) 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 5 is available 2024-02-10T07:40:58.062 INFO:teuthology.orchestra.run.smithi022.stdout:Setting orchestrator backend to cephadm... 2024-02-10T07:40:58.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:58 smithi022 ceph-mon[25704]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T07:40:58.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:58 smithi022 ceph-mon[25704]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T07:40:58.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:58 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:40:59.891 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: [10/Feb/2024:07:40:57] ENGINE Bus STARTING 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: [10/Feb/2024:07:40:57] ENGINE Serving on https://172.21.15.22:7150 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: [10/Feb/2024:07:40:58] ENGINE Serving on http://172.21.15.22:8765 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: [10/Feb/2024:07:40:58] ENGINE Bus STARTED 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: mgrmap e8: y(active, since 2s) 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:40:59.892 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:40:59 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:00.414 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout value unchanged 2024-02-10T07:41:00.414 INFO:teuthology.orchestra.run.smithi022.stdout:Generating ssh key... 2024-02-10T07:41:00.764 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:00 smithi022 ceph-mon[25704]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-mon[25704]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: Generating public/private rsa key pair. 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: Your identification has been saved in /tmp/tmpunmkt1i7/key. 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: Your public key has been saved in /tmp/tmpunmkt1i7/key.pub. 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: The key fingerprint is: 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: SHA256:0c70V39iPWODBsND7GITX4SeaJ10qMzYz9WA2s4APRk ceph-80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:41:01.902 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: The key's randomart image is: 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: +---[RSA 3072]----+ 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | .E+.=. | 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | . *o* + | 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | B /== o .| 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | . ^ X+..+.| 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | S @ oo+=+| 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | =.o..=| 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | | 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | | 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: | | 2024-02-10T07:41:01.903 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:01 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: +----[SHA256]-----+ 2024-02-10T07:41:02.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:02 smithi022 ceph-mon[25704]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:02.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:02 smithi022 ceph-mon[25704]: Generating ssh key... 2024-02-10T07:41:02.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:02 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:02.769 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:02 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:41:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:41:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:Adding key to root@localhost authorized_keys... 2024-02-10T07:41:03.327 INFO:teuthology.orchestra.run.smithi022.stdout:Adding host smithi022... 2024-02-10T07:41:03.574 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:03 smithi022 ceph-mon[25704]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:05.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:05 smithi022 ceph-mon[25704]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi022", "addr": "172.21.15.22", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:06.947 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:06 smithi022 ceph-mon[25704]: Deploying cephadm binary to smithi022 2024-02-10T07:41:09.018 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Added host 'smithi022' with addr '172.21.15.22' 2024-02-10T07:41:09.018 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mon service... 2024-02-10T07:41:09.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:09 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:09.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:09 smithi022 ceph-mon[25704]: Added host smithi022 2024-02-10T07:41:09.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:09 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:10.394 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-02-10T07:41:10.395 INFO:teuthology.orchestra.run.smithi022.stdout:Deploying unmanaged mgr service... 2024-02-10T07:41:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:10 smithi022 ceph-mon[25704]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:10 smithi022 ceph-mon[25704]: Saving service mon spec with placement count:5 2024-02-10T07:41:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:10 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:12.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:12 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:12.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:12 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:12.327 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:12 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:12.329 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-02-10T07:41:13.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:13 smithi022 ceph-mon[25704]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:13.559 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:13 smithi022 ceph-mon[25704]: Saving service mgr spec with placement count:2 2024-02-10T07:41:13.560 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:13 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/911584425' entity='client.admin' 2024-02-10T07:41:15.120 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling the dashboard module... 2024-02-10T07:41:15.677 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:15 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2315987505' entity='client.admin' 2024-02-10T07:41:15.678 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:15 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:16.765 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:16 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1475501718' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-10T07:41:16.766 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:16 smithi022 ceph-mon[25704]: from='mgr.14120 172.21.15.22:0/2384918947' entity='mgr.y' 2024-02-10T07:41:16.766 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:16 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setuser ceph since I am not root 2024-02-10T07:41:16.766 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:16 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setgroup ceph since I am not root 2024-02-10T07:41:17.385 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:17.091+0000 7efde48c1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:41:17.386 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:17.187+0000 7efde48c1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:41:17.642 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1475501718' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-10T07:41:17.643 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-mon[25704]: mgrmap e9: y(active, since 20s) 2024-02-10T07:41:17.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:17.890+0000 7efde48c1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:41:18.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:17.985+0000 7efde48c1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:41:18.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.131+0000 7efde48c1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:41:18.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.229+0000 7efde48c1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:41:18.315 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.314+0000 7efde48c1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:41:18.588 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3630805930' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-10T07:41:18.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:41:18.694 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "available": true, 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for the mgr to restart... 2024-02-10T07:41:18.695 INFO:teuthology.orchestra.run.smithi022.stdout:Waiting for mgr epoch 9... 2024-02-10T07:41:18.891 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.588+0000 7efde48c1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:41:18.891 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.702+0000 7efde48c1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:41:18.891 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.793+0000 7efde48c1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:41:18.891 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:18.890+0000 7efde48c1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:41:19.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:19 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:19.075+0000 7efde48c1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:41:20.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:20 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:20.071+0000 7efde48c1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:41:20.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:20 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:20.205+0000 7efde48c1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:41:20.836 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:20 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:20.542+0000 7efde48c1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:41:20.836 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:20 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:20.635+0000 7efde48c1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:41:20.836 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:20 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:20.834+0000 7efde48c1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:41:21.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:21 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:21.024+0000 7efde48c1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:41:21.581 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:21 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:21.317+0000 7efde48c1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:41:21.581 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:21 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:21.579+0000 7efde48c1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:41:21.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:21 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:21.821+0000 7efde48c1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:41:22.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:22 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:22.644+0000 7efde48c1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:41:23.549 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:23.226+0000 7efde48c1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:41:23.550 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:23.326+0000 7efde48c1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:41:23.550 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:41:23.548+0000 7efde48c1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:41:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: Active manager daemon y restarted 2024-02-10T07:41:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: Activating manager daemon y 2024-02-10T07:41:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: osdmap e3: 0 total, 0 up, 0 in 2024-02-10T07:41:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: mgrmap e10: y(active, starting, since 0.00307556s) 2024-02-10T07:41:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: Manager daemon y is now available 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:23.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:24.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:41:24.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:41:24.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: [10/Feb/2024:07:41:24] ENGINE Bus STARTING 2024-02-10T07:41:24.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: [10/Feb/2024:07:41:24] ENGINE Serving on http://172.21.15.22:8765 2024-02-10T07:41:24.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: [10/Feb/2024:07:41:24] ENGINE Serving on https://172.21.15.22:7150 2024-02-10T07:41:24.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: [10/Feb/2024:07:41:24] ENGINE Bus STARTED 2024-02-10T07:41:24.935 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:24 smithi022 ceph-mon[25704]: mgrmap e11: y(active, since 1.00579s) 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout { 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout "initialized": true 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout } 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:mgr epoch 9 is available 2024-02-10T07:41:25.188 INFO:teuthology.orchestra.run.smithi022.stdout:Generating a dashboard self-signed certificate... 2024-02-10T07:41:25.869 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:25 smithi022 ceph-mon[25704]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-10T07:41:25.870 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:25 smithi022 ceph-mon[25704]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-10T07:41:26.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:26 smithi022 ceph-mon[25704]: mgrmap e12: y(active, since 2s) 2024-02-10T07:41:26.858 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:26.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:26 smithi022 ceph-mon[25704]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:26.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:26.859 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-02-10T07:41:27.144 INFO:teuthology.orchestra.run.smithi022.stdout:Creating initial admin user... 2024-02-10T07:41:28.592 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$K1gTOatDzBw9HUqT0DOD5uvbEJB7T2GI9PAVDqp4EwZq8k.DIvaqy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1707550888, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-10T07:41:28.592 INFO:teuthology.orchestra.run.smithi022.stdout:Fetching dashboard port number... 2024-02-10T07:41:28.925 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:28.925 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:28 smithi022 ceph-mon[25704]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:28.925 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:30.168 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stdout 8443 2024-02-10T07:41:30.179 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-02-10T07:41:30.179 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout disabled 2024-02-10T07:41:30.187 INFO:teuthology.orchestra.run.smithi022.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-02-10T07:41:30.187 INFO:teuthology.orchestra.run.smithi022.stdout:systemctl: stdout inactive 2024-02-10T07:41:30.187 INFO:teuthology.orchestra.run.smithi022.stdout:firewalld.service is not enabled 2024-02-10T07:41:30.187 INFO:teuthology.orchestra.run.smithi022.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout:Ceph Dashboard is now available at: 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: URL: https://smithi022.front.sepia.ceph.com:8443/ 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: User: admin 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: Password: wf1ebzvq1l 2024-02-10T07:41:30.189 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:30.190 INFO:teuthology.orchestra.run.smithi022.stdout:Saving cluster configuration to /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config directory 2024-02-10T07:41:30.190 INFO:teuthology.orchestra.run.smithi022.stdout:Enabling autotune for osd_memory_target 2024-02-10T07:41:30.577 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:30 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/605911978' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-10T07:41:31.640 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:31 smithi022 ceph-mon[25704]: mgrmap e13: y(active, since 6s) 2024-02-10T07:41:33.199 INFO:teuthology.orchestra.run.smithi022.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-02-10T07:41:33.199 INFO:teuthology.orchestra.run.smithi022.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-10T07:41:33.199 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.199 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:41:33.199 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout:Or, if you are only running a single cluster on this host: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout:Please consider enabling telemetry to help improve Ceph: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: ceph telemetry on 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout:For more information see: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:33.200 INFO:teuthology.orchestra.run.smithi022.stdout:Bootstrap complete. 2024-02-10T07:41:33.244 INFO:tasks.cephadm:Fetching config... 2024-02-10T07:41:33.244 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:41:33.244 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-10T07:41:33.260 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-10T07:41:33.260 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:41:33.260 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-10T07:41:33.314 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-10T07:41:33.315 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:41:33.315 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-10T07:41:33.381 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-10T07:41:33.382 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:41:33.382 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-10T07:41:33.436 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-10T07:41:33.437 DEBUG:teuthology.orchestra.run.smithi022:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T07:41:33.516 INFO:teuthology.orchestra.run.smithi022.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:41:33.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T07:41:33.593 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:41:33.612 DEBUG:teuthology.orchestra.run.smithi081:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-10T07:41:33.657 INFO:teuthology.orchestra.run.smithi081.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDOyg0wO+FKanKgE/b2JUga3Ybwdxp7E5QO5RKgpOQr1+7NFXftJ1uUX4fNapNTX907LHhLZrGhytKNXrVwO7TfObfo8BX31trXkdoUQc0RH3IUIPGD5F2iyfwTSPfLIFmElKhcv4EyGo/WftTjFo9dhtkTdCRZJPS/ceD4RBQhc6P3H6Y2uYEID18Qg/PVwfHrlmAnSiwqblEL15oVw79Xd/5KCz4qyrmNPx+E77ezUU9Ezf00lITCAfivscHxgmtPPkJoBH0e7MPL+sa1alyuMwwGUxAzHLaK4MRQZWi1elmC4AoS3bt8KdH8NSpLVzUqn8+WlNuPUgNMtHhz8AlonO3xrIaP9JjX49F39mCZgAfGwFnG4D79q5SRYunmqe+opRtFHl1ltsdF+ebrBj7QZJn2ieKQz7Rq1F6eQgo30sm4mjnoOrtIpZGhZQvJoJTlrFSF8jVFIRmPoHxM6ow2cyJeNhuV+DstvOEMQO/PWdCWoSY8JDMWUbns02pnwXk= ceph-80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:41:33.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-10T07:41:33.775 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:33 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1863712455' entity='client.admin' 2024-02-10T07:41:33.911 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi022", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:36.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:36 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:37.317 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-10T07:41:37.317 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-10T07:41:37.566 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:37.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:37 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2101358782' entity='client.admin' 2024-02-10T07:41:37.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:37.656 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:40.719 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi027 2024-02-10T07:41:40.719 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:41:40.719 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.conf 2024-02-10T07:41:40.739 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:41:40.739 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:41:40.797 INFO:tasks.cephadm:Adding host smithi027 to orchestrator... 2024-02-10T07:41:40.798 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch host add smithi027 2024-02-10T07:41:41.035 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:41:41.283 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:41.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:41.284 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:40 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:42.170 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:41 smithi022 ceph-mon[25704]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:41:43.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:43 smithi022 ceph-mon[25704]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:45.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:45 smithi022 ceph-mon[25704]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:45.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:45 smithi022 ceph-mon[25704]: Deploying cephadm binary to smithi027 2024-02-10T07:41:47.430 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi027' with addr '172.21.15.27' 2024-02-10T07:41:47.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:47 smithi022 ceph-mon[25704]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:47.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:47 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:48.301 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T07:41:48.545 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:48.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:48 smithi022 ceph-mon[25704]: Added host smithi027 2024-02-10T07:41:48.669 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:48 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:48.670 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:48 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:49.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:49 smithi022 ceph-mon[25704]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:49.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:50.388 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:41:50.388 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}] 2024-02-10T07:41:51.023 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi081 2024-02-10T07:41:51.024 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:41:51.024 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.conf 2024-02-10T07:41:51.042 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:41:51.042 DEBUG:teuthology.orchestra.run.smithi081:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:41:51.099 INFO:tasks.cephadm:Adding host smithi081 to orchestrator... 2024-02-10T07:41:51.099 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch host add smithi081 2024-02-10T07:41:51.338 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:51.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:51 smithi022 ceph-mon[25704]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:51.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:51 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:51.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:51 smithi022 ceph-mon[25704]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:41:53.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:53 smithi022 ceph-mon[25704]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:41:54.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:41:54.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: Deploying cephadm binary to smithi081 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:41:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:55 smithi022 ceph-mon[25704]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:41:57.525 INFO:teuthology.orchestra.run.smithi022.stdout:Added host 'smithi081' with addr '172.21.15.81' 2024-02-10T07:41:57.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:57 smithi022 ceph-mon[25704]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:57.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:57.866 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:58.139 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch host ls --format=json 2024-02-10T07:41:58.380 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:41:58.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:58.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:58 smithi022 ceph-mon[25704]: Added host smithi081 2024-02-10T07:41:58.793 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:41:58.794 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:41:59.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:59 smithi022 ceph-mon[25704]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:41:59.874 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:41:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:00.194 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:42:00.195 INFO:teuthology.orchestra.run.smithi022.stdout:[{"addr": "172.21.15.22", "hostname": "smithi022", "labels": [], "status": ""}, {"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "172.21.15.81", "hostname": "smithi081", "labels": [], "status": ""}] 2024-02-10T07:42:00.794 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-10T07:42:00.794 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd crush tunables default 2024-02-10T07:42:01.029 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:42:01.412 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:01 smithi022 ceph-mon[25704]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:01.412 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:01 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:01.413 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:01 smithi022 ceph-mon[25704]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:42:03.132 INFO:teuthology.orchestra.run.smithi022.stderr:adjusted tunables profile to default 2024-02-10T07:42:03.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:03 smithi022 ceph-mon[25704]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:03.438 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:03 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T07:42:03.807 INFO:tasks.cephadm:Adding mon.a on smithi022 2024-02-10T07:42:03.807 INFO:tasks.cephadm:Adding mon.b on smithi027 2024-02-10T07:42:03.807 INFO:tasks.cephadm:Adding mon.c on smithi081 2024-02-10T07:42:03.807 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply mon '3;smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c' 2024-02-10T07:42:04.043 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:42:04.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:04.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:04.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:42:04.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:04 smithi022 ceph-mon[25704]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:05.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:05.648 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled mon update... 2024-02-10T07:42:06.230 DEBUG:teuthology.orchestra.run.smithi027:mon.b> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.b.service 2024-02-10T07:42:06.233 DEBUG:teuthology.orchestra.run.smithi081:mon.c> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.c.service 2024-02-10T07:42:06.235 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:42:06.235 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:42:06.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:06.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:06.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c;count:3 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:06.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:06 smithi022 ceph-mon[25704]: Deploying daemon mon.c on smithi081 2024-02-10T07:42:06.970 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:08 smithi022 ceph-mon[25704]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:10.074 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:42:10.074 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":1,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","modified":"2024-02-10T07:40:17.512626Z","created":"2024-02-10T07:40:17.512626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-10T07:42:10.076 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 1 2024-02-10T07:42:10.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 systemd[1]: Starting Ceph mon.c for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:42:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:10 smithi022 ceph-mon[25704]: from='client.? 172.21.15.81:0/1458677205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:10.965 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 podman[30195]: 2024-02-10 07:42:10.780559469 +0000 UTC m=+0.113770406 container create 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:42:10.965 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 podman[30195]: 2024-02-10 07:42:10.68411384 +0000 UTC m=+0.017324786 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:42:10.966 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 podman[30195]: 2024-02-10 07:42:10.966059251 +0000 UTC m=+0.299270190 container init 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c, org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240131, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux ) 2024-02-10T07:42:11.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 podman[30195]: 2024-02-10 07:42:10.971241228 +0000 UTC m=+0.304452166 container start 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 ceph-mon[30233]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 ceph-mon[30233]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 ceph-mon[30233]: pidfile_write: ignore empty --pid-file 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:10 smithi081 ceph-mon[30233]: load: jerasure load: lrc 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: RocksDB version: 7.9.2 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Git sha 0 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: DB SUMMARY 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: DB Session ID: XEGWQ5ZQBK0XENJLBHX3 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: CURRENT file: CURRENT 2024-02-10T07:42:11.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.error_if_exists: 0 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.create_if_missing: 0 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.paranoid_checks: 1 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.env: 0x560bdbfaeae0 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.info_log: 0x560bde60b8c0 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.statistics: (nil) 2024-02-10T07:42:11.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.use_fsync: 0 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_log_file_size: 0 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T07:42:11.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_fallocate: 1 2024-02-10T07:42:11.351 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T07:42:11.351 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T07:42:11.351 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.use_direct_reads: 0 2024-02-10T07:42:11.351 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.db_log_dir: 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.wal_dir: 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.write_buffer_manager: 0x560bde5df400 2024-02-10T07:42:11.352 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.unordered_write: 0 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.row_cache: None 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.wal_filter: None 2024-02-10T07:42:11.353 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.two_write_queues: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.wal_compression: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.atomic_flush: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.log_readahead_size: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T07:42:11.354 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_background_jobs: 2 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_background_compactions: -1 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_subcompactions: 1 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_open_files: -1 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T07:42:11.355 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_background_flushes: -1 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Compression algorithms supported: 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kZSTD supported: 0 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kXpressCompression supported: 0 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kZlibCompression supported: 1 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kSnappyCompression supported: 1 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kLZ4Compression supported: 1 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: kBZip2Compression supported: 0 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T07:42:11.356 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.merge_operator: 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_filter: None 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x560bde60b9c0) 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: cache_index_and_filter_blocks: 1 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: pin_top_level_index_and_filter: 1 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: index_type: 0 2024-02-10T07:42:11.357 INFO:journalctl@ceph.mon.c.smithi081.stdout: data_block_index_type: 0 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: index_shortening: 1 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: checksum: 4 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: no_block_cache: 0 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_cache: 0x560bde60f090 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_cache_name: BinnedLRUCache 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_cache_options: 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: capacity : 536870912 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: num_shard_bits : 4 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: strict_capacity_limit : 0 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: high_pri_pool_ratio: 0.000 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_cache_compressed: (nil) 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: persistent_cache: (nil) 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_size: 4096 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_size_deviation: 10 2024-02-10T07:42:11.358 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_restart_interval: 16 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: index_block_restart_interval: 1 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: metadata_block_size: 4096 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: partition_filters: 0 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: use_delta_encoding: 1 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: filter_policy: bloomfilter 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: whole_key_filtering: 1 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: verify_compression: 0 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: read_amp_bytes_per_bit: 0 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: format_version: 5 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: enable_index_compression: 1 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: block_align: 0 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: max_auto_readahead_size: 262144 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: prepopulate_block_cache: 0 2024-02-10T07:42:11.359 INFO:journalctl@ceph.mon.c.smithi081.stdout: initial_auto_readahead_size: 8192 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression: NoCompression 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.num_levels: 7 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T07:42:11.360 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:42:11.361 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T07:42:11.362 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T07:42:11.363 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.table_properties_collectors: 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.inplace_update_support: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.bloom_locality: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.max_successive_merges: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.ttl: 2592000 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T07:42:11.364 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enable_blob_files: false 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.min_blob_size: 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 78623fdc-48b4-4f14-96f6-4155222cd6b3 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550931001161, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-10T07:42:11.365 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550931001661, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707550931, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "78623fdc-48b4-4f14-96f6-4155222cd6b3", "db_session_id": "XEGWQ5ZQBK0XENJLBHX3", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550931001773, "job": 1, "event": "recovery_finished"} 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x560bde700000 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: DB pointer 0x560bde6ea000 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: ** DB Stats ** 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T07:42:11.366 INFO:journalctl@ceph.mon.c.smithi081.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: ** Compaction Stats [default] ** 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: ** Compaction Stats [default] ** 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T07:42:11.367 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Cumulative compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Interval compaction: 0.00 GB write, 0.53 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Block cache BinnedLRUCache@0x560bde60f090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-10T07:42:11.368 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: using public_addr v2:172.21.15.81:0/0 -> [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: starting mon.c rank -1 at public addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] at bind addrs [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(???) e0 preinit fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).mds e1 new map 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).mds e1 print_map 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: e1 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: legacy client fscid: -1 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout: No filesystems configured 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T07:42:11.369 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:11.370 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Deploying cephadm binary to smithi081 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.371 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Added host smithi081 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.372 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.373 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c;count:3 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: Deploying daemon mon.c on smithi081 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.81:0/1458677205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 ceph-mon[30233]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 bash[30195]: 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da 2024-02-10T07:42:11.374 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:11 smithi081 systemd[1]: Started Ceph mon.c for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:42:11.598 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:42:11.599 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:42:11.833 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:42:16.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:16.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: mon.a calling monitor election 2024-02-10T07:42:16.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: mon.c calling monitor election 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: fsmap 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: mgrmap e13: y(active, since 52s) 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: overall HEALTH_OK 2024-02-10T07:42:16.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:16.196 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:16 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: mon.a calling monitor election 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: mon.c calling monitor election 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:16.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: fsmap 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: mgrmap e13: y(active, since 52s) 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: overall HEALTH_OK 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:16.599 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:16 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:16.744 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:42:16.744 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":2,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","modified":"2024-02-10T07:42:11.027473Z","created":"2024-02-10T07:40:17.512626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T07:42:16.746 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 2 2024-02-10T07:42:17.286 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:17 smithi081 ceph-mon[30233]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:17.286 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:17 smithi081 ceph-mon[30233]: Deploying daemon mon.b on smithi027 2024-02-10T07:42:17.286 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:17 smithi081 ceph-mon[30233]: from='client.? 172.21.15.81:0/942106247' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:17.286 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:17 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:17.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:17 smithi022 ceph-mon[25704]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:17.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:17 smithi022 ceph-mon[25704]: Deploying daemon mon.b on smithi027 2024-02-10T07:42:17.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:17 smithi022 ceph-mon[25704]: from='client.? 172.21.15.81:0/942106247' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:17.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:17 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:18.313 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-10T07:42:18.313 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph mon dump -f json 2024-02-10T07:42:18.558 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:42:19.316 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:19 smithi081 ceph-mon[30233]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:19.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:19 smithi022 ceph-mon[25704]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:19.799 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 podman[29806]: 2024-02-10 07:42:19.599099375 +0000 UTC m=+0.018033482 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:42:20.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 podman[29806]: 2024-02-10 07:42:19.885379704 +0000 UTC m=+0.304313798 container init 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 podman[29806]: 2024-02-10 07:42:19.891206542 +0000 UTC m=+0.310140636 container start 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, ceph=True) 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: set uid:gid to 167:167 (ceph:ceph) 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: ceph version 19.0.0-1361-g290212a1 (290212a1bfd33563ba76ee5b7f0b944fdb6c5d90) squid (dev), process ceph-mon, pid 7 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pidfile_write: ignore empty --pid-file 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: load: jerasure load: lrc 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: RocksDB version: 7.9.2 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Git sha 0 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Compile date 2024-02-09 17:35:33 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: DB SUMMARY 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: DB Session ID: FI1TCN735DS30O8MXFXX 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: CURRENT file: CURRENT 2024-02-10T07:42:20.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: IDENTITY file: IDENTITY 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.error_if_exists: 0 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.create_if_missing: 0 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.paranoid_checks: 1 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.flush_verify_memtable_count: 1 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.env: 0x564c2c624ae0 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.fs: PosixFileSystem 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.info_log: 0x564c2e2ef8c0 2024-02-10T07:42:20.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_file_opening_threads: 16 2024-02-10T07:42:20.378 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.statistics: (nil) 2024-02-10T07:42:20.378 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.use_fsync: 0 2024-02-10T07:42:20.378 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_log_file_size: 0 2024-02-10T07:42:20.378 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-10T07:42:20.378 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.log_file_time_to_roll: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.keep_log_file_num: 1000 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.recycle_log_file_num: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_fallocate: 1 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_mmap_reads: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_mmap_writes: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.use_direct_reads: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.create_missing_column_families: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.db_log_dir: 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.wal_dir: 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.table_cache_numshardbits: 6 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.WAL_ttl_seconds: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.WAL_size_limit_MB: 0 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-10T07:42:20.379 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.is_fd_close_on_exec: 1 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.advise_random_on_open: 1 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.db_write_buffer_size: 0 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.write_buffer_manager: 0x564c2e2c3400 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.use_adaptive_mutex: 0 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.rate_limiter: (nil) 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.wal_recovery_mode: 2 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enable_thread_tracking: 0 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enable_pipelined_write: 0 2024-02-10T07:42:20.380 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.unordered_write: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.row_cache: None 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.wal_filter: None 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_ingest_behind: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.two_write_queues: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.manual_wal_flush: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.wal_compression: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.atomic_flush: 0 2024-02-10T07:42:20.381 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.persist_stats_to_disk: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.write_dbid_to_manifest: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.log_readahead_size: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.best_efforts_recovery: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.allow_data_in_errors: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.db_host_id: __hostname__ 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enforce_single_del_contracts: true 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_background_jobs: 2 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_background_compactions: -1 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_subcompactions: 1 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-10T07:42:20.382 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.delayed_write_rate : 16777216 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_total_wal_size: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.stats_dump_period_sec: 600 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.stats_persist_period_sec: 600 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_open_files: -1 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bytes_per_sync: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.wal_bytes_per_sync: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.strict_bytes_per_sync: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_readahead_size: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_background_flushes: -1 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Compression algorithms supported: 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kZSTD supported: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kXpressCompression supported: 0 2024-02-10T07:42:20.383 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kLZ4HCCompression supported: 1 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kZlibCompression supported: 1 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kSnappyCompression supported: 1 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kLZ4Compression supported: 1 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: kBZip2Compression supported: 0 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Fast CRC32 supported: Supported on x86 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: DMutex implementation: pthread_mutex_t 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.merge_operator: 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_filter: None 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_filter_factory: None 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.sst_partitioner_factory: None 2024-02-10T07:42:20.384 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.memtable_factory: SkipListFactory 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.table_factory: BlockBasedTable 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564c2e2ef9c0) 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: cache_index_and_filter_blocks: 1 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: pin_top_level_index_and_filter: 1 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_type: 0 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: data_block_index_type: 0 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_shortening: 1 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: checksum: 4 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: no_block_cache: 0 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache: 0x564c2e2f3090 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_name: BinnedLRUCache 2024-02-10T07:42:20.385 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_options: 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: capacity : 536870912 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: num_shard_bits : 4 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: strict_capacity_limit : 0 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: high_pri_pool_ratio: 0.000 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_cache_compressed: (nil) 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: persistent_cache: (nil) 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_size: 4096 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_size_deviation: 10 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_restart_interval: 16 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: index_block_restart_interval: 1 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: metadata_block_size: 4096 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: partition_filters: 0 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: use_delta_encoding: 1 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: filter_policy: bloomfilter 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: whole_key_filtering: 1 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: verify_compression: 0 2024-02-10T07:42:20.386 INFO:journalctl@ceph.mon.b.smithi027.stdout: read_amp_bytes_per_bit: 0 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: format_version: 5 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: enable_index_compression: 1 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: block_align: 0 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: max_auto_readahead_size: 262144 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: prepopulate_block_cache: 0 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: initial_auto_readahead_size: 8192 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout: num_file_reads_for_auto_readahead: 2 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.write_buffer_size: 33554432 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_write_buffer_number: 2 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression: NoCompression 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression: Disabled 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.prefix_extractor: nullptr 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.num_levels: 7 2024-02-10T07:42:20.387 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.window_bits: -14 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.level: 32767 2024-02-10T07:42:20.388 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.strategy: 0 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.enabled: false 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.target_file_size_base: 67108864 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.target_file_size_multiplier: 1 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-10T07:42:20.389 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.arena_block_size: 1048576 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.disable_auto_compactions: 0 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-10T07:42:20.390 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.table_properties_collectors: 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.inplace_update_support: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.inplace_update_num_locks: 10000 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.memtable_huge_page_size: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.bloom_locality: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.max_successive_merges: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.optimize_filters_for_hits: 0 2024-02-10T07:42:20.391 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.paranoid_file_checks: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.force_consistency_checks: 1 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.report_bg_io_stats: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.ttl: 2592000 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.periodic_compaction_seconds: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enable_blob_files: false 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.min_blob_size: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_file_size: 268435456 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_compression_type: NoCompression 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.enable_blob_garbage_collection: false 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.blob_file_starting_level: 0 2024-02-10T07:42:20.392 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9ae1abe0-996d-400d-948a-fac9b10c748a 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550939922081, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550939922612, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1707550939, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9ae1abe0-996d-400d-948a-fac9b10c748a", "db_session_id": "FI1TCN735DS30O8MXFXX", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: EVENT_LOG_v1 {"time_micros": 1707550939922716, "job": 1, "event": "recovery_finished"} 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564c2e3e4000 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: DB pointer 0x564c2e3ce000 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** DB Stats ** 2024-02-10T07:42:20.393 INFO:journalctl@ceph.mon.b.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** Compaction Stats [default] ** 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** Compaction Stats [default] ** 2024-02-10T07:42:20.394 INFO:journalctl@ceph.mon.b.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Interval compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Block cache BinnedLRUCache@0x564c2e2f3090#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-02-10T07:42:20.395 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: using public_addr v2:172.21.15.27:0/0 -> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: starting mon.b rank -1 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(???) e0 preinit fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).mds e1 new map 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).mds e1 print_map 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: e1 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: legacy client fscid: -1 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout: No filesystems configured 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-10T07:42:20.396 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.397 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi081", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Deploying cephadm binary to smithi081 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.398 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Added host smithi081 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1571082289' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:20.399 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi081", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.400 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Saving service mon spec with placement smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c;count:3 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Deploying daemon mon.c on smithi081 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.? 172.21.15.81:0/1458677205' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.a calling monitor election 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.401 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.c calling monitor election 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: monmap e2: 2 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: fsmap 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mgrmap e13: y(active, since 52s) 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: overall HEALTH_OK 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:20.402 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: Deploying daemon mon.b on smithi027 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='client.? 172.21.15.81:0/942106247' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 ceph-mon[29844]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 bash[29806]: 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 2024-02-10T07:42:20.403 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:19 smithi027 systemd[1]: Started Ceph mon.b for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:42:25.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: mon.a calling monitor election 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: mon.c calling monitor election 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: fsmap 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: mgrmap e13: y(active, since 61s) 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: mon.b (rank 2) addr [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] is down (out of quorum) 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:25 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: mon.a calling monitor election 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: mon.c calling monitor election 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: fsmap 2024-02-10T07:42:25.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: mgrmap e13: y(active, since 61s) 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: mon.b (rank 2) addr [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] is down (out of quorum) 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:25.446 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:25 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:26.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:26 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:26.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:26 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:26.385 INFO:teuthology.orchestra.run.smithi081.stdout: 2024-02-10T07:42:26.385 INFO:teuthology.orchestra.run.smithi081.stdout:{"epoch":3,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","modified":"2024-02-10T07:42:19.947301Z","created":"2024-02-10T07:40:17.512626Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:3300","nonce":0},{"type":"v1","addr":"172.21.15.22:6789","nonce":0}]},"addr":"172.21.15.22:6789/0","public_addr":"172.21.15.22:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:3300","nonce":0},{"type":"v1","addr":"172.21.15.81:6789","nonce":0}]},"addr":"172.21.15.81:6789/0","public_addr":"172.21.15.81:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-10T07:42:26.388 INFO:teuthology.orchestra.run.smithi081.stderr:dumped monmap epoch 3 2024-02-10T07:42:26.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:26.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.068 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-10T07:42:27.068 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph config generate-minimal-conf 2024-02-10T07:42:27.320 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:42:27.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:27.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:42:27.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: mon.a calling monitor election 2024-02-10T07:42:27.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: mon.c calling monitor election 2024-02-10T07:42:27.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: fsmap 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: mgrmap e13: y(active, since 61s) 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-02-10T07:42:27.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: mon.b (rank 2) addr [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] is down (out of quorum) 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:27.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mon.b calling monitor election 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mon.b calling monitor election 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mon.c calling monitor election 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mon.a calling monitor election 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:28.077 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: fsmap 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: mgrmap e13: y(active, since 63s) 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Cluster is now healthy 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: overall HEALTH_OK 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.078 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:42:28.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:28.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:28.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:28.079 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:27 smithi022 ceph-mon[25704]: Reconfiguring daemon mon.a on smithi022 2024-02-10T07:42:28.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mon.b calling monitor election 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mon.b calling monitor election 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mon.c calling monitor election 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mon.a calling monitor election 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: fsmap 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: mgrmap e13: y(active, since 63s) 2024-02-10T07:42:28.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Cluster is now healthy 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: overall HEALTH_OK 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:28.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:28.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:27 smithi081 ceph-mon[30233]: Reconfiguring daemon mon.a on smithi022 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mon.b calling monitor election 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mon.b calling monitor election 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mon.c calling monitor election 2024-02-10T07:42:28.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mon.a calling monitor election 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: monmap e3: 3 mons at {a=[v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0],b=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],c=[v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: fsmap 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: osdmap e4: 0 total, 0 up, 0 in 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: mgrmap e13: y(active, since 63s) 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Cluster is now healthy 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: overall HEALTH_OK 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Reconfiguring mon.a (unknown last config time)... 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:28.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:27 smithi027 ceph-mon[29844]: Reconfiguring daemon mon.a on smithi022 2024-02-10T07:42:29.177 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:28 smithi022 ceph-mon[25704]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:29.177 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:29.177 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:29.308 INFO:teuthology.orchestra.run.smithi022.stdout:# minimal ceph.conf for 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:29.308 INFO:teuthology.orchestra.run.smithi022.stdout:[global] 2024-02-10T07:42:29.308 INFO:teuthology.orchestra.run.smithi022.stdout: fsid = 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:42:29.308 INFO:teuthology.orchestra.run.smithi022.stdout: mon_host = [v2:172.21.15.22:3300/0,v1:172.21.15.22:6789/0] [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] 2024-02-10T07:42:29.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:28 smithi081 ceph-mon[30233]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:29.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:28 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:29.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:28 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:29.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:28 smithi027 ceph-mon[29844]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:29.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:28 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:42:29.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:28 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:30.164 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:29 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2609070530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:30.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:29 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2609070530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:30.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:29 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2609070530' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:30.667 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-10T07:42:30.667 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:42:30.667 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T07:42:30.699 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:42:30.699 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:30.769 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:42:30.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T07:42:30.801 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:42:30.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:30.872 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:42:30.872 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.conf 2024-02-10T07:42:30.906 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:42:30.906 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:42:30.975 INFO:tasks.cephadm:Adding mgr.y on smithi022 2024-02-10T07:42:30.976 INFO:tasks.cephadm:Adding mgr.x on smithi027 2024-02-10T07:42:30.976 INFO:tasks.cephadm:Adding mgr.z on smithi081 2024-02-10T07:42:30.976 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply mgr '3;smithi022=y;smithi027=x;smithi081=z' 2024-02-10T07:42:31.246 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:31.488 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:31 smithi081 ceph-mon[30233]: Reconfiguring daemon mon.b on smithi027 2024-02-10T07:42:31.519 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:31.519 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.519 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:42:31.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:31.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:31.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:31.520 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:31 smithi027 ceph-mon[29844]: Reconfiguring daemon mon.b on smithi027 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: Reconfiguring mon.b (monmap changed)... 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:31.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:31 smithi022 ceph-mon[25704]: Reconfiguring daemon mon.b on smithi027 2024-02-10T07:42:33.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:33.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:42:33.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:33.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:33.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:33.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:32 smithi081 ceph-mon[30233]: Reconfiguring daemon mon.c on smithi081 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:33.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:33.125 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:32 smithi027 ceph-mon[29844]: Reconfiguring daemon mon.c on smithi081 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: Reconfiguring mon.c (monmap changed)... 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:33.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:32 smithi022 ceph-mon[25704]: Reconfiguring daemon mon.c on smithi081 2024-02-10T07:42:33.866 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled mgr update... 2024-02-10T07:42:34.418 DEBUG:teuthology.orchestra.run.smithi027:mgr.x> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x.service 2024-02-10T07:42:34.422 DEBUG:teuthology.orchestra.run.smithi081:mgr.z> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.z.service 2024-02-10T07:42:34.424 INFO:tasks.cephadm:Deploying OSDs... 2024-02-10T07:42:34.425 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:42:34.425 DEBUG:teuthology.orchestra.run.smithi022:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:42:34.439 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:42:34.440 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/nvme1n1 2024-02-10T07:42:34.494 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/nvme1n1 2024-02-10T07:42:34.494 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:34.494 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 764 Links: 1 Device type: 103,2 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:41:37.310781098 +0000 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:39:16.058486936 +0000 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:39:16.058486936 +0000 2024-02-10T07:42:34.495 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:39:16.054486988 +0000 2024-02-10T07:42:34.495 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T07:42:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:42:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:42:34.561 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000123613 s, 4.1 MB/s 2024-02-10T07:42:34.562 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T07:42:34.617 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/nvme2n1 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/nvme2n1 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 774 Links: 1 Device type: 103,4 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:41:37.320780972 +0000 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:39:16.213484896 +0000 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:39:16.213484896 +0000 2024-02-10T07:42:34.671 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:39:16.202485041 +0000 2024-02-10T07:42:34.672 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.728 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:34.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:34.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:34.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:34.729 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.737 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:42:34.737 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:42:34.737 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000115449 s, 4.4 MB/s 2024-02-10T07:42:34.738 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.756 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:34.757 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.757 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:34.757 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:34.757 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:34.757 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.793 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/nvme3n1 2024-02-10T07:42:34.848 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/nvme3n1 2024-02-10T07:42:34.848 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:41:37.329780858 +0000 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:39:16.360482961 +0000 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:39:16.360482961 +0000 2024-02-10T07:42:34.849 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:39:16.358482987 +0000 2024-02-10T07:42:34.849 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:34.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:34.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:34.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:42:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:42:34.914 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000109221 s, 4.7 MB/s 2024-02-10T07:42:34.916 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T07:42:34.971 DEBUG:teuthology.orchestra.run.smithi022:> stat /dev/nvme4n1 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout: File: /dev/nvme4n1 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Access: 2024-02-10 07:41:37.338780744 +0000 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Modify: 2024-02-10 07:39:16.521480842 +0000 2024-02-10T07:42:35.025 INFO:teuthology.orchestra.run.smithi022.stdout:Change: 2024-02-10 07:39:16.521480842 +0000 2024-02-10T07:42:35.026 INFO:teuthology.orchestra.run.smithi022.stdout: Birth: 2024-02-10 07:39:16.514480934 +0000 2024-02-10T07:42:35.026 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T07:42:35.090 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records in 2024-02-10T07:42:35.090 INFO:teuthology.orchestra.run.smithi022.stderr:1+0 records out 2024-02-10T07:42:35.090 INFO:teuthology.orchestra.run.smithi022.stderr:512 bytes copied, 0.000122137 s, 4.2 MB/s 2024-02-10T07:42:35.091 DEBUG:teuthology.orchestra.run.smithi022:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T07:42:35.146 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:42:35.146 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:42:35.165 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:42:35.165 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/nvme1n1 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/nvme1n1 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:41:52.523672332 +0000 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:39:17.788753218 +0000 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:39:17.788753218 +0000 2024-02-10T07:42:35.224 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:39:17.776753797 +0000 2024-02-10T07:42:35.224 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T07:42:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:42:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:42:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000265063 s, 1.9 MB/s 2024-02-10T07:42:35.299 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T07:42:35.357 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/nvme2n1 2024-02-10T07:42:35.413 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/nvme2n1 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:41:52.535672017 +0000 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:39:17.967744566 +0000 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:39:17.967744566 +0000 2024-02-10T07:42:35.414 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:39:17.955745146 +0000 2024-02-10T07:42:35.414 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T07:42:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:42:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:42:35.486 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000216589 s, 2.4 MB/s 2024-02-10T07:42:35.487 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T07:42:35.544 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/nvme3n1 2024-02-10T07:42:35.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:35 smithi081 ceph-mon[30233]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:35.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:35 smithi081 ceph-mon[30233]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=y;smithi027=x;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:35.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:35 smithi081 ceph-mon[30233]: Saving service mgr spec with placement smithi022=y;smithi027=x;smithi081=z;count:3 2024-02-10T07:42:35.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:35 smithi081 ceph-mon[30233]: Deploying daemon mgr.z on smithi081 2024-02-10T07:42:35.601 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/nvme3n1 2024-02-10T07:42:35.601 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.601 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-10T07:42:35.601 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.601 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.602 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:41:52.546671728 +0000 2024-02-10T07:42:35.602 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:39:18.142739732 +0000 2024-02-10T07:42:35.602 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:39:18.142739732 +0000 2024-02-10T07:42:35.602 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:39:18.130740049 +0000 2024-02-10T07:42:35.602 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T07:42:35.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:35 smithi027 ceph-mon[29844]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:35.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:35 smithi027 ceph-mon[29844]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=y;smithi027=x;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:35.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:35 smithi027 ceph-mon[29844]: Saving service mgr spec with placement smithi022=y;smithi027=x;smithi081=z;count:3 2024-02-10T07:42:35.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:35 smithi027 ceph-mon[29844]: Deploying daemon mgr.z on smithi081 2024-02-10T07:42:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:42:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:42:35.654 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000290372 s, 1.8 MB/s 2024-02-10T07:42:35.656 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T07:42:35.712 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/nvme4n1 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/nvme4n1 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2024-02-10 07:41:52.558671412 +0000 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2024-02-10 07:39:18.277736175 +0000 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2024-02-10 07:39:18.277736175 +0000 2024-02-10T07:42:35.769 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2024-02-10 07:39:18.266736464 +0000 2024-02-10T07:42:35.769 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T07:42:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2024-02-10T07:42:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2024-02-10T07:42:35.839 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000205253 s, 2.5 MB/s 2024-02-10T07:42:35.840 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T07:42:35.896 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:42:35.896 DEBUG:teuthology.orchestra.run.smithi081:> dd if=/scratch_devs of=/dev/stdout 2024-02-10T07:42:35.912 DEBUG:teuthology.misc:devs=['/dev/nvme1n1', '/dev/nvme2n1', '/dev/nvme3n1', '/dev/nvme4n1'] 2024-02-10T07:42:35.912 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/nvme1n1 2024-02-10T07:42:35.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:35 smithi022 ceph-mon[25704]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:35.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:35 smithi022 ceph-mon[25704]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi022=y;smithi027=x;smithi081=z", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:35.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:35 smithi022 ceph-mon[25704]: Saving service mgr spec with placement smithi022=y;smithi027=x;smithi081=z;count:3 2024-02-10T07:42:35.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:35 smithi022 ceph-mon[25704]: Deploying daemon mgr.z on smithi081 2024-02-10T07:42:35.966 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/nvme1n1 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 762 Links: 1 Device type: 103,2 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:42:02.248375276 +0000 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:39:19.504482292 +0000 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:39:19.504482292 +0000 2024-02-10T07:42:35.967 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:39:19.492482521 +0000 2024-02-10T07:42:35.967 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/nvme1n1 of=/dev/null count=1 2024-02-10T07:42:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:42:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:42:36.036 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000233237 s, 2.2 MB/s 2024-02-10T07:42:36.037 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/nvme1n1 2024-02-10T07:42:36.093 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/nvme2n1 2024-02-10T07:42:36.148 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/nvme2n1 2024-02-10T07:42:36.148 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 775 Links: 1 Device type: 103,4 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:42:02.258375087 +0000 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:39:19.656479387 +0000 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:39:19.656479387 +0000 2024-02-10T07:42:36.149 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:39:19.644479616 +0000 2024-02-10T07:42:36.149 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/nvme2n1 of=/dev/null count=1 2024-02-10T07:42:36.214 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:42:36.214 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:42:36.214 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000191962 s, 2.7 MB/s 2024-02-10T07:42:36.215 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/nvme2n1 2024-02-10T07:42:36.271 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/nvme3n1 2024-02-10T07:42:36.326 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 podman[32133]: 2024-02-10 07:42:36.133370066 +0000 UTC m=+0.017122853 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:42:36.327 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/nvme3n1 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 787 Links: 1 Device type: 103,6 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:42:02.269374878 +0000 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:39:19.801476616 +0000 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:39:19.801476616 +0000 2024-02-10T07:42:36.328 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:39:19.790476827 +0000 2024-02-10T07:42:36.328 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/nvme3n1 of=/dev/null count=1 2024-02-10T07:42:36.395 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:42:36.395 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:42:36.395 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000184636 s, 2.8 MB/s 2024-02-10T07:42:36.396 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/nvme3n1 2024-02-10T07:42:36.452 DEBUG:teuthology.orchestra.run.smithi081:> stat /dev/nvme4n1 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout: File: /dev/nvme4n1 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 0 Blocks: 0 IO Block: 4096 block special file 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 799 Links: 1 Device type: 103,8 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0660/brw-rw----) Uid: ( 0/ root) Gid: ( 6/ disk) 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:fixed_disk_device_t:s0 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2024-02-10 07:42:02.280374670 +0000 2024-02-10T07:42:36.509 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2024-02-10 07:39:19.952473731 +0000 2024-02-10T07:42:36.510 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2024-02-10 07:39:19.952473731 +0000 2024-02-10T07:42:36.510 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: 2024-02-10 07:39:19.944473884 +0000 2024-02-10T07:42:36.510 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd if=/dev/nvme4n1 of=/dev/null count=1 2024-02-10T07:42:36.576 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2024-02-10T07:42:36.576 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2024-02-10T07:42:36.576 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes copied, 0.000211779 s, 2.4 MB/s 2024-02-10T07:42:36.577 DEBUG:teuthology.orchestra.run.smithi081:> ! mount | grep -v devtmpfs | grep -q /dev/nvme4n1 2024-02-10T07:42:36.633 INFO:tasks.cephadm:Deploying osd.0 on smithi022 with /dev/nvme4n1... 2024-02-10T07:42:36.633 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T07:42:36.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 podman[32133]: 2024-02-10 07:42:36.407740301 +0000 UTC m=+0.291493080 container init c138fc9611131149c144735f6742638539d5fbc720bdaf9d8599312e275e6551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=main-290212a, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131) 2024-02-10T07:42:36.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 podman[32133]: 2024-02-10 07:42:36.413105019 +0000 UTC m=+0.296857798 container start c138fc9611131149c144735f6742638539d5fbc720bdaf9d8599312e275e6551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z, ceph=True, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240131) 2024-02-10T07:42:36.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 ceph-mgr[32257]: -- 172.21.15.81:0/3394638607 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55d5e6b0c1a0 con 0x55d5e6ae2c00 2024-02-10T07:42:36.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 bash[32133]: c138fc9611131149c144735f6742638539d5fbc720bdaf9d8599312e275e6551 2024-02-10T07:42:36.848 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 systemd[1]: Started Ceph mgr.z for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:42:36.876 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:42:37.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:36 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:36.962+0000 7fc8df367200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:42:37.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:37.054+0000 7fc8df367200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:37.672 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:37.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:37.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:37.673 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:37 smithi022 ceph-mon[25704]: Deploying daemon mgr.x on smithi027 2024-02-10T07:42:37.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:37.686+0000 7fc8df367200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:42:37.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:37.778+0000 7fc8df367200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:42:37.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:37.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-mon[30233]: Deploying daemon mgr.x on smithi027 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:37 smithi027 ceph-mon[29844]: Deploying daemon mgr.x on smithi027 2024-02-10T07:42:38.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:37 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:37.917+0000 7fc8df367200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:42:38.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.011+0000 7fc8df367200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:42:38.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.093+0000 7fc8df367200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:42:38.640 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.352+0000 7fc8df367200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:42:38.640 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.460+0000 7fc8df367200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:42:38.640 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.546+0000 7fc8df367200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:42:38.640 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.639+0000 7fc8df367200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr: stderr: 10+0 records in 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:10+0 records out 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00950016 s, 1.1 GB/s 2024-02-10T07:42:38.755 INFO:teuthology.orchestra.run.smithi022.stderr:--> Zapping successful for: 2024-02-10T07:42:38.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:38 smithi027 podman[31294]: 2024-02-10 07:42:38.811266198 +0000 UTC m=+0.134489264 container create df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=main-290212a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.license=GPLv2) 2024-02-10T07:42:39.097 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:38 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:38.812+0000 7fc8df367200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:42:39.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 podman[31294]: 2024-02-10 07:42:39.013849534 +0000 UTC m=+0.337072603 container init df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=main-290212a, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-10T07:42:39.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 podman[31294]: 2024-02-10 07:42:39.019199204 +0000 UTC m=+0.342422272 container start df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.2) 2024-02-10T07:42:39.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mgr[31332]: -- 172.21.15.27:0/320072020 <== mon.2 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56112e96e1a0 con 0x56112e945000 2024-02-10T07:42:39.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 bash[31294]: df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 2024-02-10T07:42:39.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 systemd[1]: Started Ceph mgr.x for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:42:39.387 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi022:/dev/nvme4n1 2024-02-10T07:42:39.623 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:39 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.793 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:39.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:39.596+0000 7fb6bc1f8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:42:39.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:39.694+0000 7fb6bc1f8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:42:39.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:39.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:39.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:39 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:40.097 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:39.792+0000 7fc8df367200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:42:40.097 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:39 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:39.943+0000 7fc8df367200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:42:40.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:40.389+0000 7fb6bc1f8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:42:40.485 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:40.484+0000 7fb6bc1f8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:42:40.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:40 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:40.287+0000 7fc8df367200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:42:40.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:40 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:40.388+0000 7fc8df367200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:42:40.737 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:40.637+0000 7fb6bc1f8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:42:40.737 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:40.737+0000 7fb6bc1f8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:42:41.061 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:40 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:40.600+0000 7fc8df367200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:42:41.061 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:40 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:40.783+0000 7fc8df367200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:42:41.092 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:40.822+0000 7fb6bc1f8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:42:41.092 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:41.092+0000 7fb6bc1f8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:42:41.316 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:41.060+0000 7fc8df367200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:42:41.316 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:41.315+0000 7fc8df367200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:42:41.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:41.204+0000 7fb6bc1f8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:42:41.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:41.292+0000 7fb6bc1f8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:42:41.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:41.548+0000 7fc8df367200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:42:41.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:41.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:42:41.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.631 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:42:41.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:42:41.632 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:41.389+0000 7fb6bc1f8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:41.570+0000 7fb6bc1f8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:42:41.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:42.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:42.337+0000 7fc8df367200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:42:42.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: Reconfiguring daemon mgr.y on smithi022 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:42.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: Reconfiguring daemon mgr.y on smithi022 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:42.795 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:42 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:42.592+0000 7fb6bc1f8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:42.726+0000 7fb6bc1f8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: Reconfiguring mgr.y (unknown last config time)... 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: Reconfiguring daemon mgr.y on smithi022 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi022:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:42.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:42:42.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:42 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:43.209 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:42.901+0000 7fc8df367200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:42:43.209 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:42 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:42.997+0000 7fc8df367200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:42:43.209 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:42:43.208+0000 7fc8df367200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:42:43.362 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:43.072+0000 7fb6bc1f8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:42:43.362 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:43.163+0000 7fb6bc1f8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:42:43.362 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:43.362+0000 7fb6bc1f8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:43.551+0000 7fb6bc1f8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: Standby manager daemon z started 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:42:43.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:43.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:43.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: Standby manager daemon z started 2024-02-10T07:42:43.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:42:43.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:43.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:42:43.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:43 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:43.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:43.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: Standby manager daemon z started 2024-02-10T07:42:43.852 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:42:43.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:43.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:42:43.853 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:43 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/3960608057' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:44.118 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:43 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:43.842+0000 7fb6bc1f8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:42:44.119 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:44.118+0000 7fb6bc1f8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:42:44.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:44.358+0000 7fb6bc1f8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:42:44.815 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T07:42:44.815 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:42:44.815 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:44.815 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]: dispatch 2024-02-10T07:42:44.815 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]': finished 2024-02-10T07:42:44.816 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:42:44.816 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:44 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]: dispatch 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]': finished 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:42:44.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:44 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: mgrmap e14: y(active, since 79s), standbys: z 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]: dispatch 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4163125377' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1fe9ab54-a7ac-4c67-973d-3bd111153324"}]': finished 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: osdmap e5: 1 total, 0 up, 1 in 2024-02-10T07:42:44.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:44 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:42:45.558 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:45 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:45.185+0000 7fb6bc1f8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:42:45.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:45 smithi081 ceph-mon[30233]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:45.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:45 smithi027 ceph-mon[29844]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:45.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:45 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:45.777+0000 7fb6bc1f8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:42:45.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:45 smithi022 ceph-mon[25704]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:46.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:45 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:45.877+0000 7fb6bc1f8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:42:46.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:42:46.097+0000 7fb6bc1f8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:42:46.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4178418637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:42:46.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: Standby manager daemon x started 2024-02-10T07:42:46.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:42:46.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:46.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:42:46.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:46 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4178418637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: Standby manager daemon x started 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:42:46.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:46 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4178418637' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: Standby manager daemon x started 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:42:46.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:46 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/1161058943' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:42:47.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:47 smithi081 ceph-mon[30233]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:47.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:47 smithi081 ceph-mon[30233]: mgrmap e15: y(active, since 83s), standbys: z, x 2024-02-10T07:42:47.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:47 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:42:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:47 smithi027 ceph-mon[29844]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:47 smithi027 ceph-mon[29844]: mgrmap e15: y(active, since 83s), standbys: z, x 2024-02-10T07:42:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:47 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:42:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:47 smithi022 ceph-mon[25704]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:47 smithi022 ceph-mon[25704]: mgrmap e15: y(active, since 83s), standbys: z, x 2024-02-10T07:42:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:47 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:42:48.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:48 smithi027 ceph-mon[29844]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:48.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:48 smithi022 ceph-mon[25704]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:49.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:48 smithi081 ceph-mon[30233]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:51.662 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:51 smithi022 ceph-mon[25704]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:51.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:51 smithi081 ceph-mon[30233]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:51 smithi027 ceph-mon[29844]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:52.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:52 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:42:52.564 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:52 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:52.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:52 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:42:52.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:52 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:52 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-10T07:42:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:52 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:42:53.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:53 smithi022 ceph-mon[25704]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:53.738 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:53 smithi022 ceph-mon[25704]: Deploying daemon osd.0 on smithi022 2024-02-10T07:42:53.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:53 smithi081 ceph-mon[30233]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:53.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:53 smithi081 ceph-mon[30233]: Deploying daemon osd.0 on smithi022 2024-02-10T07:42:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:53 smithi027 ceph-mon[29844]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:53 smithi027 ceph-mon[29844]: Deploying daemon osd.0 on smithi022 2024-02-10T07:42:55.619 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:55 smithi022 ceph-mon[25704]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:55.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:55 smithi081 ceph-mon[30233]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:55.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:55 smithi027 ceph-mon[29844]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:57.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:57 smithi081 ceph-mon[30233]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:57.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:57 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:57 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:57 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:57.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:57 smithi027 ceph-mon[29844]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:57.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:57 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:57 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:57 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:57.884 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:57 smithi022 ceph-mon[25704]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:57.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:42:57.885 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:42:59.812 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:42:59 smithi022 ceph-mon[25704]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:59.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:42:59 smithi081 ceph-mon[30233]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:42:59.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:42:59 smithi027 ceph-mon[29844]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.099 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:00 smithi022 ceph-mon[25704]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:00 smithi027 ceph-mon[29844]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:43:01.171 INFO:teuthology.orchestra.run.smithi022.stdout:Created osd(s) 0 on host 'smithi022' 2024-02-10T07:43:01.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:01.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:01.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:01.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:01.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:00 smithi081 ceph-mon[30233]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-10T07:43:01.760 DEBUG:teuthology.orchestra.run.smithi022:osd.0> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.0.service 2024-02-10T07:43:01.763 INFO:tasks.cephadm:Deploying osd.1 on smithi027 with /dev/nvme4n1... 2024-02-10T07:43:01.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T07:43:02.021 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:43:02.062 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:02.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:01 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:01 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-10T07:43:02.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: osdmap e6: 1 total, 0 up, 1 in 2024-02-10T07:43:02.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]: dispatch 2024-02-10T07:43:02.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:02.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:02.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:01 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:02.444 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0[43841]: 2024-02-10T07:43:02.149+0000 7f1048f92700 -1 osd.0 0 waiting for initial osdmap 2024-02-10T07:43:02.444 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0[43841]: 2024-02-10T07:43:02.151+0000 7f104057c700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:02.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:02 smithi022 ceph-mon[25704]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.055 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:02 smithi027 ceph-mon[29844]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi022", "root=default"]}]': finished 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: osdmap e7: 1 total, 0 up, 1 in 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.249 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:02 smithi081 ceph-mon[30233]: from='osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017]' entity='osd.0' 2024-02-10T07:43:03.979 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: purged_snaps scrub starts 2024-02-10T07:43:03.979 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: purged_snaps scrub ok 2024-02-10T07:43:03.979 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.980 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017] boot 2024-02-10T07:43:03.980 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:43:03.980 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011526 s, 910 MB/s 2024-02-10T07:43:03.980 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: purged_snaps scrub starts 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: purged_snaps scrub ok 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017] boot 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:43:04.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: purged_snaps scrub starts 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: purged_snaps scrub ok 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: osd.0 [v2:172.21.15.22:6802/3108645017,v1:172.21.15.22:6803/3108645017] boot 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: osdmap e8: 1 total, 1 up, 1 in 2024-02-10T07:43:04.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:43:04.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi027:/dev/nvme4n1 2024-02-10T07:43:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:43:05.330 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:05.330 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:43:05.330 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:05.331 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.427 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:05.428 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-10T07:43:05.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: osdmap e9: 1 total, 1 up, 1 in 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:05.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:06.337 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:06 smithi027 ceph-mon[29844]: Detected new or changed devices on smithi022 2024-02-10T07:43:06.338 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:06 smithi027 ceph-mon[29844]: Adjusting osd_memory_target on smithi022 to 17093M 2024-02-10T07:43:06.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:06 smithi022 ceph-mon[25704]: Detected new or changed devices on smithi022 2024-02-10T07:43:06.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:06 smithi022 ceph-mon[25704]: Adjusting osd_memory_target on smithi022 to 17093M 2024-02-10T07:43:06.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:06 smithi081 ceph-mon[30233]: Detected new or changed devices on smithi022 2024-02-10T07:43:06.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:06 smithi081 ceph-mon[30233]: Adjusting osd_memory_target on smithi022 to 17093M 2024-02-10T07:43:07.426 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:07 smithi027 ceph-mon[29844]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:07.426 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:07 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:07.426 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:07 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:07.426 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:07 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:07.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:07 smithi022 ceph-mon[25704]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:07.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:07 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:07.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:07 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:07.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:07 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:07.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:07 smithi081 ceph-mon[30233]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:07.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:07 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:07.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:07 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:07.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:07 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:08.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:08 smithi027 ceph-mon[29844]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:08.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:08 smithi022 ceph-mon[25704]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:08.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:08 smithi081 ceph-mon[30233]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:09.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:09 smithi027 ceph-mon[29844]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:09.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:09 smithi022 ceph-mon[25704]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:09.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:09 smithi081 ceph-mon[30233]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:10 smithi081 ceph-mon[30233]: from='client.? 172.21.15.27:0/2603957235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:10 smithi081 ceph-mon[30233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:10 smithi081 ceph-mon[30233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]': finished 2024-02-10T07:43:10.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:10 smithi081 ceph-mon[30233]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:43:10.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:10 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:10 smithi027 ceph-mon[29844]: from='client.? 172.21.15.27:0/2603957235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:10 smithi027 ceph-mon[29844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:10 smithi027 ceph-mon[29844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]': finished 2024-02-10T07:43:10.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:10 smithi027 ceph-mon[29844]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:43:10.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:10 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:10 smithi022 ceph-mon[25704]: from='client.? 172.21.15.27:0/2603957235' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:10 smithi022 ceph-mon[25704]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]: dispatch 2024-02-10T07:43:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:10 smithi022 ceph-mon[25704]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "67a22425-d450-48ef-a0fe-c6310951cf06"}]': finished 2024-02-10T07:43:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:10 smithi022 ceph-mon[25704]: osdmap e10: 2 total, 1 up, 2 in 2024-02-10T07:43:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:10 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:11.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:11 smithi081 ceph-mon[30233]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:11.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.27:0/1866822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:11 smithi027 ceph-mon[29844]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:11 smithi027 ceph-mon[29844]: from='client.? 172.21.15.27:0/1866822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:11 smithi022 ceph-mon[25704]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:11 smithi022 ceph-mon[25704]: from='client.? 172.21.15.27:0/1866822612' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:13.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:13 smithi081 ceph-mon[30233]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:13 smithi027 ceph-mon[29844]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:13 smithi022 ceph-mon[25704]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:15.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:15 smithi027 ceph-mon[29844]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:15.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:15 smithi081 ceph-mon[30233]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:15.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:15 smithi022 ceph-mon[25704]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:17.828 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:17 smithi027 ceph-mon[29844]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:17.828 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:17 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:43:17.828 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:17 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:17 smithi081 ceph-mon[30233]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:17 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:43:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:17 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:17.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:17 smithi022 ceph-mon[25704]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:17.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:17 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-10T07:43:17.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:17 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:18.744 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:18 smithi027 ceph-mon[29844]: Deploying daemon osd.1 on smithi027 2024-02-10T07:43:18.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:18 smithi081 ceph-mon[30233]: Deploying daemon osd.1 on smithi027 2024-02-10T07:43:18.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:18 smithi022 ceph-mon[25704]: Deploying daemon osd.1 on smithi027 2024-02-10T07:43:19.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:19 smithi027 ceph-mon[29844]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:19.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:19 smithi081 ceph-mon[30233]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:19.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:19 smithi022 ceph-mon[25704]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:21.844 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:21 smithi027 ceph-mon[29844]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:21.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:21 smithi081 ceph-mon[30233]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:21.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:21 smithi022 ceph-mon[25704]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:23.767 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:23 smithi027 ceph-mon[29844]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:23.768 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:23 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.768 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:23 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.768 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:23 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:23.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:23 smithi081 ceph-mon[30233]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:23.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:23 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:23 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:23 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:23 smithi022 ceph-mon[25704]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:23 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:25.585 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:25 smithi027 ceph-mon[29844]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:25.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:25 smithi081 ceph-mon[30233]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:25.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:25 smithi022 ceph-mon[25704]: pgmap v60: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:26.413 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 1 on host 'smithi027' 2024-02-10T07:43:26.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:26 smithi027 ceph-mon[29844]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:26 smithi027 ceph-mon[29844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:26.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:26 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:26.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:26 smithi081 ceph-mon[30233]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:26 smithi081 ceph-mon[30233]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:26 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:26.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:26 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:26.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:26 smithi022 ceph-mon[25704]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:26 smithi022 ceph-mon[25704]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-10T07:43:26.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:26.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:26 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.035 DEBUG:teuthology.orchestra.run.smithi027:osd.1> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.1.service 2024-02-10T07:43:27.038 INFO:tasks.cephadm:Deploying osd.2 on smithi081 with /dev/nvme4n1... 2024-02-10T07:43:27.038 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph-volume lvm zap /dev/nvme4n1 2024-02-10T07:43:27.285 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:43:27.686 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:27.686 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:43:27.686 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:43:27.686 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.686 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:27.687 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.687 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.687 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:27 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.801 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:27.801 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.802 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: pgmap v61: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:27.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-10T07:43:27.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: osdmap e11: 2 total, 1 up, 2 in 2024-02-10T07:43:27.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:27.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2024-02-10T07:43:27.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:27.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:27 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:28.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1[34423]: 2024-02-10T07:43:27.988+0000 7fd1f1ab2700 -1 osd.1 0 waiting for initial osdmap 2024-02-10T07:43:28.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1[34423]: 2024-02-10T07:43:27.992+0000 7fd1e989d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:43:28.800 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:28 smithi081 ceph-mon[30233]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-02-10T07:43:28.800 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:28 smithi081 ceph-mon[30233]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:43:28.800 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:28 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.800 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:28 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.800 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:28 smithi081 ceph-mon[30233]: from='osd.1 ' entity='osd.1' 2024-02-10T07:43:28.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:28 smithi027 ceph-mon[29844]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-02-10T07:43:28.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:28 smithi027 ceph-mon[29844]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:43:28.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:28 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:28 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.873 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:28 smithi027 ceph-mon[29844]: from='osd.1 ' entity='osd.1' 2024-02-10T07:43:28.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:28 smithi022 ceph-mon[25704]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2024-02-10T07:43:28.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:28 smithi022 ceph-mon[25704]: osdmap e12: 2 total, 1 up, 2 in 2024-02-10T07:43:28.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:28 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:28.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:28 smithi022 ceph-mon[25704]: from='osd.1 ' entity='osd.1' 2024-02-10T07:43:29.146 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping: /dev/nvme4n1 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr:--> --destroy was not specified, but zapping a whole device will remove the partition table 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/nvme4n1 bs=1M count=10 conv=fsync 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr: stderr: 10+0 records in 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr:10+0 records out 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.014284 s, 734 MB/s 2024-02-10T07:43:29.147 INFO:teuthology.orchestra.run.smithi081.stderr:--> Zapping successful for: 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: purged_snaps scrub starts 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: purged_snaps scrub ok 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668] boot 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:43:29.745 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:29.771 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch daemon add osd smithi081:/dev/nvme4n1 2024-02-10T07:43:29.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: purged_snaps scrub starts 2024-02-10T07:43:29.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: purged_snaps scrub ok 2024-02-10T07:43:29.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:29.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:29.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668] boot 2024-02-10T07:43:29.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:43:29.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: purged_snaps scrub starts 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: purged_snaps scrub ok 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: pgmap v64: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: osd.1 [v2:172.21.15.27:6800/3302204668,v1:172.21.15.27:6801/3302204668] boot 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: osdmap e13: 2 total, 2 up, 2 in 2024-02-10T07:43:29.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:43:30.004 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:43:31.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:31.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:43:31.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:31 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:31.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:43:31.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:31 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: osdmap e14: 2 total, 2 up, 2 in 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:31.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:31.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:31.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:31.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:31 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:32.740 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:32 smithi081 ceph-mon[30233]: Detected new or changed devices on smithi027 2024-02-10T07:43:32.740 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:32 smithi081 ceph-mon[30233]: Adjusting osd_memory_target on smithi027 to 17088M 2024-02-10T07:43:32.740 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:32.740 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:32.740 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:32 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:32.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:32 smithi027 ceph-mon[29844]: Detected new or changed devices on smithi027 2024-02-10T07:43:32.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:32 smithi027 ceph-mon[29844]: Adjusting osd_memory_target on smithi027 to 17088M 2024-02-10T07:43:32.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:32.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:32.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:32 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:32 smithi022 ceph-mon[25704]: Detected new or changed devices on smithi027 2024-02-10T07:43:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:32 smithi022 ceph-mon[25704]: Adjusting osd_memory_target on smithi027 to 17088M 2024-02-10T07:43:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-10T07:43:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-10T07:43:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:32 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:33.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:33 smithi022 ceph-mon[25704]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:33.845 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:33 smithi022 ceph-mon[25704]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:33.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:33 smithi081 ceph-mon[30233]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:33.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:33 smithi081 ceph-mon[30233]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:33.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:33 smithi027 ceph-mon[29844]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:33.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:33 smithi027 ceph-mon[29844]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi081:/dev/nvme4n1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:43:34.806 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:34 smithi081 ceph-mon[30233]: from='client.? 172.21.15.81:0/3055585105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.806 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:34 smithi081 ceph-mon[30233]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.806 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:34 smithi081 ceph-mon[30233]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]': finished 2024-02-10T07:43:34.806 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:34 smithi081 ceph-mon[30233]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:43:34.806 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:34 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:34 smithi027 ceph-mon[29844]: from='client.? 172.21.15.81:0/3055585105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:34 smithi027 ceph-mon[29844]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:34 smithi027 ceph-mon[29844]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]': finished 2024-02-10T07:43:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:34 smithi027 ceph-mon[29844]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:43:34.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:34 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:34.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:34 smithi022 ceph-mon[25704]: from='client.? 172.21.15.81:0/3055585105' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:34 smithi022 ceph-mon[25704]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]: dispatch 2024-02-10T07:43:34.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:34 smithi022 ceph-mon[25704]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "36678171-675b-4433-b1f9-19cb96ed3d2f"}]': finished 2024-02-10T07:43:34.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:34 smithi022 ceph-mon[25704]: osdmap e15: 3 total, 2 up, 3 in 2024-02-10T07:43:34.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:34 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:35.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:35 smithi081 ceph-mon[30233]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:35.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:35 smithi027 ceph-mon[29844]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:35.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:35 smithi022 ceph-mon[25704]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:36.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:36 smithi081 ceph-mon[30233]: from='client.? 172.21.15.81:0/816810043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:36.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:36 smithi027 ceph-mon[29844]: from='client.? 172.21.15.81:0/816810043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:36.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:36 smithi022 ceph-mon[25704]: from='client.? 172.21.15.81:0/816810043' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-10T07:43:37.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:37 smithi081 ceph-mon[30233]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:37.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:37 smithi027 ceph-mon[29844]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:37.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:37 smithi022 ceph-mon[25704]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:39.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:39 smithi081 ceph-mon[30233]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:39.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:39 smithi027 ceph-mon[29844]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:39.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:39 smithi022 ceph-mon[25704]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:41.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:41 smithi081 ceph-mon[30233]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:41.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:43:41.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:41 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:41 smithi027 ceph-mon[29844]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:43:41.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:41 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:41.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:41 smithi022 ceph-mon[25704]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:41.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-10T07:43:41.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:41 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:42.797 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:42 smithi081 ceph-mon[30233]: Deploying daemon osd.2 on smithi081 2024-02-10T07:43:42.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:42 smithi027 ceph-mon[29844]: Deploying daemon osd.2 on smithi081 2024-02-10T07:43:42.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:42 smithi022 ceph-mon[25704]: Deploying daemon osd.2 on smithi081 2024-02-10T07:43:43.634 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:43 smithi081 ceph-mon[30233]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:43.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:43 smithi027 ceph-mon[29844]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:43.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:43 smithi022 ceph-mon[25704]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:45.794 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:45 smithi081 ceph-mon[30233]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:45.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:45 smithi027 ceph-mon[29844]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:45.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:45 smithi022 ceph-mon[25704]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:47.792 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:47 smithi081 ceph-mon[30233]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:47.792 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:47 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:47.792 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:47 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:47.792 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:47 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:47 smithi027 ceph-mon[29844]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:47 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:47 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:47.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:47 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:47 smithi022 ceph-mon[25704]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:47 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:47 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:47.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:47 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.736 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:49.737 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.737 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.737 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:49.737 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:49.737 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:49 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:49.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:49.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:49.876 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:49 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: pgmap v77: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:49.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:49 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:50.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:50 smithi081 ceph-mon[30233]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:50 smithi081 ceph-mon[30233]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:50 smithi027 ceph-mon[29844]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:50 smithi027 ceph-mon[29844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:50 smithi022 ceph-mon[25704]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:50 smithi022 ceph-mon[25704]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-10T07:43:50.945 INFO:teuthology.orchestra.run.smithi081.stdout:Created osd(s) 2 on host 'smithi081' 2024-02-10T07:43:51.609 DEBUG:teuthology.orchestra.run.smithi081:osd.2> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.2.service 2024-02-10T07:43:51.612 INFO:tasks.cephadm:Waiting for 3 OSDs to come up... 2024-02-10T07:43:51.612 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd stat -f json 2024-02-10T07:43:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:43:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:43:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:51.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:51 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:51.861 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:51 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:51.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: pgmap v78: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:51.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-10T07:43:51.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: osdmap e16: 3 total, 2 up, 3 in 2024-02-10T07:43:51.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:51.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]: dispatch 2024-02-10T07:43:51.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:51.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:51 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:43:52.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2[35491]: 2024-02-10T07:43:52.024+0000 7f17369b6700 -1 osd.2 0 waiting for initial osdmap 2024-02-10T07:43:52.348 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2[35491]: 2024-02-10T07:43:52.029+0000 7f172dfa0700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-02-10T07:43:52.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-mon[30233]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-02-10T07:43:52.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-mon[30233]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:43:52.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:52 smithi081 ceph-mon[30233]: from='osd.2 ' entity='osd.2' 2024-02-10T07:43:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:52 smithi027 ceph-mon[29844]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-02-10T07:43:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:52 smithi027 ceph-mon[29844]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:43:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:52 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:52 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:52 smithi027 ceph-mon[29844]: from='osd.2 ' entity='osd.2' 2024-02-10T07:43:52.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:52 smithi022 ceph-mon[25704]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi081", "root=default"]}]': finished 2024-02-10T07:43:52.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:52 smithi022 ceph-mon[25704]: osdmap e17: 3 total, 2 up, 3 in 2024-02-10T07:43:52.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:52 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:52 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:52.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:52 smithi022 ceph-mon[25704]: from='osd.2 ' entity='osd.2' 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: purged_snaps scrub starts 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: purged_snaps scrub ok 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935] boot 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:43:53.612 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:53 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:53.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: purged_snaps scrub starts 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: purged_snaps scrub ok 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935] boot 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:43:53.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:53 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: purged_snaps scrub starts 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: purged_snaps scrub ok 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: pgmap v81: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: osd.2 [v2:172.21.15.81:6800/1050443935,v1:172.21.15.81:6801/1050443935] boot 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: osdmap e18: 3 total, 3 up, 3 in 2024-02-10T07:43:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:53 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:43:54.121 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:43:54.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:54.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:43:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.695 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:54 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/745784409' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:43:54.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:54.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.731 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:43:54.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.732 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:54 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/745784409' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:43:54.761 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":19,"num_osds":3,"num_up_osds":3,"osd_up_since":1707551033,"num_in_osds":3,"osd_in_since":1707551014,"num_remapped_pgs":0} 2024-02-10T07:43:54.761 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: osdmap e19: 3 total, 3 up, 3 in 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-02-10T07:43:54.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:54 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/745784409' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-10T07:43:55.005 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:43:55.384 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[49994]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T07:43:55.385 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[49994]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:55.385 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[49994]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:55.385 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[49994]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:55.944 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[50015]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T07:43:55.944 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[50015]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:55.944 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[50015]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:55.944 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:43:55 smithi022 sudo[50015]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: Detected new or changed devices on smithi081 2024-02-10T07:43:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: Adjusting osd_memory_target on smithi081 to 17088M 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:43:55.945 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:55 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: Detected new or changed devices on smithi081 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: Adjusting osd_memory_target on smithi081 to 17088M 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:43:56.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:55 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.111 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: Detected new or changed devices on smithi081 2024-02-10T07:43:56.111 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.111 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: Adjusting osd_memory_target on smithi081 to 17088M 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: osdmap e20: 3 total, 3 up, 3 in 2024-02-10T07:43:56.112 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:55 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:56.374 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40087]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T07:43:56.374 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40087]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:56.374 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40087]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:56.374 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40087]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:56.874 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40090]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T07:43:56.874 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40090]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:56.874 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40090]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:56.874 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:43:56 smithi027 sudo[40090]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:56.875 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:43:56.875 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":21,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","created":"2024-02-10T07:40:28.951807+0000","modified":"2024-02-10T07:43:56.032011+0000","last_up_change":"2024-02-10T07:43:53.027732+0000","last_in_change":"2024-02-10T07:43:34.389966+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:43:53.910905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fe9ab54-a7ac-4c67-973d-3bd111153324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6803","nonce":3108645017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6805","nonce":3108645017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6809","nonce":3108645017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6807","nonce":3108645017}]},"public_addr":"172.21.15.22:6803/3108645017","cluster_addr":"172.21.15.22:6805/3108645017","heartbeat_back_addr":"172.21.15.22:6809/3108645017","heartbeat_front_addr":"172.21.15.22:6807/3108645017","state":["exists","up"]},{"osd":1,"uuid":"67a22425-d450-48ef-a0fe-c6310951cf06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6801","nonce":3302204668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6803","nonce":3302204668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6807","nonce":3302204668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6805","nonce":3302204668}]},"public_addr":"172.21.15.27:6801/3302204668","cluster_addr":"172.21.15.27:6803/3302204668","heartbeat_back_addr":"172.21.15.27:6807/3302204668","heartbeat_front_addr":"172.21.15.27:6805/3302204668","state":["exists","up"]},{"osd":2,"uuid":"36678171-675b-4433-b1f9-19cb96ed3d2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6801","nonce":1050443935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6803","nonce":1050443935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6807","nonce":1050443935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6805","nonce":1050443935}]},"public_addr":"172.21.15.81:6801/1050443935","cluster_addr":"172.21.15.81:6803/1050443935","heartbeat_back_addr":"172.21.15.81:6807/1050443935","heartbeat_front_addr":"172.21.15.81:6805/1050443935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:01.394570+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:26.927116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:51.212803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/2773249949":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4132953616":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4200425753":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/3221539624":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/3166432828":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/2917108429":"2024-02-11T07:40:56.450004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:43:57.084 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 ceph-mon[30233]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:57.084 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 ceph-mon[30233]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:43:57.084 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:57.084 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4147313864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:43:57.084 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:56 smithi081 sudo[40315]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme4n1 2024-02-10T07:43:57.084 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:56 smithi081 sudo[40315]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:57.085 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:56 smithi081 sudo[40315]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:57.085 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:56 smithi081 sudo[40315]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:57.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 ceph-mon[25704]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:57.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 ceph-mon[25704]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:43:57.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:57.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4147313864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:43:57.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40318]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme linux smart-log-add --json /dev/nvme4n1 2024-02-10T07:43:57.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40318]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:57.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40318]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:57.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40318]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:57.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:57 smithi027 ceph-mon[29844]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:57.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:57 smithi027 ceph-mon[29844]: osdmap e21: 3 total, 3 up, 3 in 2024-02-10T07:43:57.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:57 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:57.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:57 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4147313864' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:43:57.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50114]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:43:57.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50114]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:57.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50114]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:57.494 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50114]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:57.520 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-02-10T07:43:53.910905+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-02-10T07:43:57.520 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-02-10T07:43:57.759 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:43:57.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50135]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:43:57.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50135]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:57.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50135]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:57.880 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:57 smithi022 sudo[50135]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:58.143 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40321]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:43:58.143 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40321]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:58.144 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40321]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:58.144 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:57 smithi081 sudo[40321]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:58.144 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 sudo[40324]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:43:58.144 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 sudo[40324]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:58.144 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 sudo[40324]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:58.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T07:43:58.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.445 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:58 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 sudo[40324]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:58 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: mgrmap e16: y(active, since 2m), standbys: z, x 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.614 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.615 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:58.615 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:58.615 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:58.615 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:58.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 sudo[40093]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-02-10T07:43:58.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 sudo[40093]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:58.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 sudo[40093]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:58.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:58 smithi027 sudo[40093]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 sudo[40096]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 sudo[40096]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 sudo[40096]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 sudo[40096]: pam_unix(sudo:session): session closed for user root 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:59.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:43:59 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:43:59 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: pgmap v88: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:43:59.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-02-10T07:43:59.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:43:59 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:43:59.631 INFO:teuthology.orchestra.run.smithi022.stdout:pg_num: 1 2024-02-10T07:44:00.220 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi022 2024-02-10T07:44:00.220 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi022=foo.a' 2024-02-10T07:44:00.462 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:00.583 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:00 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2767046579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:44:00.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:00 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2767046579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:44:00.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:00 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2767046579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-02-10T07:44:01.587 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:01 smithi081 ceph-mon[30233]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:01.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:01 smithi027 ceph-mon[29844]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:01.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:01 smithi022 ceph-mon[25704]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:02.304 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled rgw.foo.a update... 2024-02-10T07:44:02.945 DEBUG:teuthology.orchestra.run.smithi022:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@rgw.foo.a.service 2024-02-10T07:44:02.948 INFO:tasks.cephadm:Adding prometheus.a on smithi027 2024-02-10T07:44:02.948 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply prometheus '1;smithi027=a' 2024-02-10T07:44:03.187 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.554 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:03 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:03.555 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.556 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:03 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='client.24211 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi022=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:03.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:03.625 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:03 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:04.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:04 smithi022 ceph-mon[25704]: Deploying daemon rgw.foo.a on smithi022 2024-02-10T07:44:04.534 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:04 smithi081 ceph-mon[30233]: Deploying daemon rgw.foo.a on smithi022 2024-02-10T07:44:04.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:04 smithi027 ceph-mon[29844]: Deploying daemon rgw.foo.a on smithi022 2024-02-10T07:44:04.995 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled prometheus update... 2024-02-10T07:44:05.423 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 systemd[1]: Starting Ceph rgw.foo.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:44:05.562 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:05 smithi081 ceph-mon[30233]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:05.563 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:05 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:05.564 DEBUG:teuthology.orchestra.run.smithi027:prometheus.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@prometheus.a.service 2024-02-10T07:44:05.567 INFO:tasks.cephadm:Adding node-exporter.a on smithi022 2024-02-10T07:44:05.567 INFO:tasks.cephadm:Adding node-exporter.b on smithi027 2024-02-10T07:44:05.567 INFO:tasks.cephadm:Adding node-exporter.c on smithi081 2024-02-10T07:44:05.567 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply node-exporter '3;smithi022=a;smithi027=b;smithi081=c' 2024-02-10T07:44:05.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:05 smithi022 ceph-mon[25704]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:05.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:05 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:05.694 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 podman[50648]: 2024-02-10 07:44:05.422319357 +0000 UTC m=+0.108666767 container create ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=main-290212a, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-10T07:44:05.694 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 podman[50648]: 2024-02-10 07:44:05.331672173 +0000 UTC m=+0.018019586 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:44:05.694 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 podman[50648]: 2024-02-10 07:44:05.666331289 +0000 UTC m=+0.352678699 container init ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-10T07:44:05.694 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 podman[50648]: 2024-02-10 07:44:05.671616268 +0000 UTC m=+0.357963687 container start ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-290212a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-02-10T07:44:05.804 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:05.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:05 smithi027 ceph-mon[29844]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:05.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:05 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.194 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 bash[50648]: ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 2024-02-10T07:44:06.194 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:44:05 smithi022 systemd[1]: Started Ceph rgw.foo.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: Saving service prometheus spec with placement smithi027=a;count:1 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.776 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:06 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:06.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:06.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: Saving service prometheus spec with placement smithi027=a;count:1 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:06 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: Saving service prometheus spec with placement smithi027=a;count:1 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:06.918 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:06 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:07.821 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled node-exporter update... 2024-02-10T07:44:07.821 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:07 smithi081 ceph-mon[30233]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:07.821 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:07 smithi081 ceph-mon[30233]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:07.821 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:07 smithi081 ceph-mon[30233]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:44:07.821 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:07 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:44:07.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:07 smithi027 ceph-mon[29844]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:07.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:07 smithi027 ceph-mon[29844]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:07.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:07 smithi027 ceph-mon[29844]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:44:07.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:07 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:44:07.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:07 smithi022 ceph-mon[25704]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:07.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:07 smithi022 ceph-mon[25704]: Saving service rgw.foo.a spec with placement smithi022=foo.a;count:1 2024-02-10T07:44:07.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:07 smithi022 ceph-mon[25704]: osdmap e22: 3 total, 3 up, 3 in 2024-02-10T07:44:07.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:07 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-10T07:44:08.455 DEBUG:teuthology.orchestra.run.smithi022:node-exporter.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.a.service 2024-02-10T07:44:08.458 DEBUG:teuthology.orchestra.run.smithi027:node-exporter.b> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.b.service 2024-02-10T07:44:08.460 DEBUG:teuthology.orchestra.run.smithi081:node-exporter.c> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.c.service 2024-02-10T07:44:08.463 INFO:tasks.cephadm:Adding alertmanager.a on smithi022 2024-02-10T07:44:08.463 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply alertmanager '1;smithi022=a' 2024-02-10T07:44:08.734 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:08.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:44:08.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T07:44:08.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:08.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:08.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: Checking dashboard <-> RGW credentials 2024-02-10T07:44:08.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:08 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: Checking dashboard <-> RGW credentials 2024-02-10T07:44:08.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:08 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2600277035' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: osdmap e23: 3 total, 3 up, 3 in 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: Checking dashboard <-> RGW credentials 2024-02-10T07:44:08.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:08 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:09.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: pgmap v95: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:09.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi022=a;smithi027=b;smithi081=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:09.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: Saving service node-exporter spec with placement smithi022=a;smithi027=b;smithi081=c;count:3 2024-02-10T07:44:09.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T07:44:09.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:09 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: pgmap v95: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi022=a;smithi027=b;smithi081=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: Saving service node-exporter spec with placement smithi022=a;smithi027=b;smithi081=c;count:3 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:09 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: pgmap v95: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 91 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: from='client.24232 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "3;smithi022=a;smithi027=b;smithi081=c", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: Saving service node-exporter spec with placement smithi022=a;smithi027=b;smithi081=c;count:3 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: osdmap e24: 3 total, 3 up, 3 in 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-10T07:44:09.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:09 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:10.573 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled alertmanager update... 2024-02-10T07:44:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:10 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:10 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:10 smithi081 ceph-mon[30233]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T07:44:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:10 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:10 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:10 smithi027 ceph-mon[29844]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T07:44:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:10 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:10 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-10T07:44:10.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:10 smithi022 ceph-mon[25704]: osdmap e25: 3 total, 3 up, 3 in 2024-02-10T07:44:11.198 DEBUG:teuthology.orchestra.run.smithi022:alertmanager.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@alertmanager.a.service 2024-02-10T07:44:11.201 INFO:tasks.cephadm:Adding grafana.a on smithi027 2024-02-10T07:44:11.201 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph orch apply grafana '1;smithi027=a' 2024-02-10T07:44:11.434 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:11.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: pgmap v98: 65 pgs: 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:11.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-02-10T07:44:11.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:11 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: pgmap v98: 65 pgs: 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-02-10T07:44:11.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:11 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: pgmap v98: 65 pgs: 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: osdmap e26: 3 total, 3 up, 3 in 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: from='client.24271 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi022=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: Saving service alertmanager spec with placement smithi022=a;count:1 2024-02-10T07:44:11.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:11 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:12.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:12 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:12 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:12 smithi081 ceph-mon[30233]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T07:44:12.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:12 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:12 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:12 smithi027 ceph-mon[29844]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T07:44:12.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:12 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:12 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-10T07:44:12.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:12 smithi022 ceph-mon[25704]: osdmap e27: 3 total, 3 up, 3 in 2024-02-10T07:44:13.339 INFO:teuthology.orchestra.run.smithi081.stdout:Scheduled grafana update... 2024-02-10T07:44:13.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: pgmap v101: 97 pgs: 32 unknown, 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:13.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T07:44:13.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:13 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: pgmap v101: 97 pgs: 32 unknown, 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:13 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: pgmap v101: 97 pgs: 32 unknown, 1 creating+activating, 1 active, 21 creating+peering, 42 active+clean; 578 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.5 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: osdmap e28: 3 total, 3 up, 3 in 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-10T07:44:13.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:13 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:13.953 DEBUG:teuthology.orchestra.run.smithi027:grafana.a> sudo journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@grafana.a.service 2024-02-10T07:44:13.955 INFO:tasks.cephadm:Setting up client nodes... 2024-02-10T07:44:13.956 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T07:44:14.200 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:14.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:14.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: Saving service grafana spec with placement smithi027=a;count:1 2024-02-10T07:44:14.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.580 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.581 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:14 smithi022 ceph-mon[25704]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: Saving service grafana spec with placement smithi027=a;count:1 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T07:44:14.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:14 smithi081 ceph-mon[30233]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi027=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: Saving service grafana spec with placement smithi027=a;count:1 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: osdmap e29: 3 total, 3 up, 3 in 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4045675754' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-10T07:44:14.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/1222326965' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-10T07:44:14.875 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:14 smithi027 ceph-mon[29844]: osdmap e30: 3 total, 3 up, 3 in 2024-02-10T07:44:15.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:15 smithi081 ceph-mon[30233]: pgmap v104: 129 pgs: 43 unknown, 1 creating+activating, 1 active, 12 creating+peering, 72 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:15.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:15 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:15 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:15 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:15 smithi027 ceph-mon[29844]: pgmap v104: 129 pgs: 43 unknown, 1 creating+activating, 1 active, 12 creating+peering, 72 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:15.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:15 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:15 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:15 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:15 smithi022 ceph-mon[25704]: pgmap v104: 129 pgs: 43 unknown, 1 creating+activating, 1 active, 12 creating+peering, 72 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:15.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:15 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:15 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:15.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:15 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:16.068 INFO:teuthology.orchestra.run.smithi022.stdout:[client.0] 2024-02-10T07:44:16.068 INFO:teuthology.orchestra.run.smithi022.stdout: key = AQBQKcdlgDIDBBAAxYYtuZb1Sqj1kWW+IrqkDw== 2024-02-10T07:44:16.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:16 smithi022 ceph-mon[25704]: Deploying daemon prometheus.a on smithi027 2024-02-10T07:44:16.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:16 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:16.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:16 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:16.750 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:44:16.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-10T07:44:16.750 DEBUG:teuthology.orchestra.run.smithi022:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-10T07:44:16.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T07:44:16.827 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:16 smithi027 ceph-mon[29844]: Deploying daemon prometheus.a on smithi027 2024-02-10T07:44:16.828 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:16 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:16.828 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:16 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:16.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:16 smithi081 ceph-mon[30233]: Deploying daemon prometheus.a on smithi027 2024-02-10T07:44:16.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:16 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:16.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:16 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/3456609957' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:17.063 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.b/config 2024-02-10T07:44:17.815 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:17 smithi027 ceph-mon[29844]: pgmap v106: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-10T07:44:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:17 smithi081 ceph-mon[30233]: pgmap v106: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-10T07:44:17.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:17 smithi022 ceph-mon[25704]: pgmap v106: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 503 B/s wr, 3 op/s 2024-02-10T07:44:19.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:19 smithi081 ceph-mon[30233]: pgmap v107: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-10T07:44:19.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:19 smithi027 ceph-mon[29844]: pgmap v107: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-10T07:44:19.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:19 smithi022 ceph-mon[25704]: pgmap v107: 129 pgs: 129 active+clean; 578 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 341 B/s wr, 2 op/s 2024-02-10T07:44:20.031 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2024-02-10T07:44:20.031 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQBUKcdltAyjARAAz7Fqy5mgnIkgH/+8FNNfZQ== 2024-02-10T07:44:20.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:20 smithi081 ceph-mon[30233]: from='client.? 172.21.15.27:0/4248603137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:20 smithi081 ceph-mon[30233]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:20 smithi081 ceph-mon[30233]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:20.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:20 smithi027 ceph-mon[29844]: from='client.? 172.21.15.27:0/4248603137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:20 smithi027 ceph-mon[29844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:20 smithi027 ceph-mon[29844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:20.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:20 smithi022 ceph-mon[25704]: from='client.? 172.21.15.27:0/4248603137' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:20 smithi022 ceph-mon[25704]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:20.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:20 smithi022 ceph-mon[25704]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:21.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:21 smithi081 ceph-mon[30233]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.1 KiB/s wr, 159 op/s 2024-02-10T07:44:21.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:21 smithi027 ceph-mon[29844]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.1 KiB/s wr, 159 op/s 2024-02-10T07:44:21.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:21 smithi022 ceph-mon[25704]: pgmap v108: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 66 KiB/s rd, 5.1 KiB/s wr, 159 op/s 2024-02-10T07:44:22.072 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:44:22.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-10T07:44:22.072 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-10T07:44:22.163 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph auth get-or-create client.2 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-10T07:44:22.407 INFO:teuthology.orchestra.run.smithi081.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.c/config 2024-02-10T07:44:23.675 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:23 smithi081 ceph-mon[30233]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:44:23.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:23 smithi027 ceph-mon[29844]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:44:23.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:23 smithi022 ceph-mon[25704]: pgmap v109: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 58 KiB/s rd, 4.5 KiB/s wr, 140 op/s 2024-02-10T07:44:24.465 INFO:teuthology.orchestra.run.smithi081.stdout:[client.2] 2024-02-10T07:44:24.465 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQBYKcdlWrl+GxAAKzYxLV5XAfoDwQrzcCsxKA== 2024-02-10T07:44:24.822 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:24 smithi081 ceph-mon[30233]: from='client.? 172.21.15.81:0/2303168122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.822 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:24 smithi081 ceph-mon[30233]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.822 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:24 smithi081 ceph-mon[30233]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:24.838 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:24 smithi027 ceph-mon[29844]: from='client.? 172.21.15.81:0/2303168122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.839 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:24 smithi027 ceph-mon[29844]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.839 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:24 smithi027 ceph-mon[29844]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:24.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:24 smithi022 ceph-mon[25704]: from='client.? 172.21.15.81:0/2303168122' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:24 smithi022 ceph-mon[25704]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-10T07:44:24.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:24 smithi022 ceph-mon[25704]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.2", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-10T07:44:25.130 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:44:25.130 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/etc/ceph/ceph.client.2.keyring 2024-02-10T07:44:25.130 DEBUG:teuthology.orchestra.run.smithi081:> sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-02-10T07:44:25.172 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-10T07:44:25.173 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-10T07:44:25.173 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph mgr dump --format=json 2024-02-10T07:44:25.422 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:25.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:25 smithi081 ceph-mon[30233]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T07:44:25.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:25 smithi027 ceph-mon[29844]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T07:44:25.883 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:25 smithi022 ceph-mon[25704]: pgmap v110: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 46 KiB/s rd, 3.6 KiB/s wr, 112 op/s 2024-02-10T07:44:26.625 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:26 smithi027 podman[41018]: 2024-02-10 07:44:26.352122729 +0000 UTC m=+0.019238432 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-02-10T07:44:27.259 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:44:27.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:27 smithi081 ceph-mon[30233]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-10T07:44:27.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:27 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/653455406' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:44:27.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:27 smithi022 ceph-mon[25704]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-10T07:44:27.850 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:27 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/653455406' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:44:27.874 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 podman[41018]: 2024-02-10 07:44:27.531097335 +0000 UTC m=+1.198213028 container create fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:27.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-mon[29844]: pgmap v111: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 42 KiB/s rd, 3.1 KiB/s wr, 100 op/s 2024-02-10T07:44:27.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/653455406' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-10T07:44:27.879 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":16,"flags":0,"active_gid":14152,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6800","nonce":1724389544},{"type":"v1","addr":"172.21.15.22:6801","nonce":1724389544}]},"active_addr":"172.21.15.22:6801/1724389544","active_change":"2024-02-10T07:41:23.562971+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.22:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1119374657}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2539440122}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":2982822982}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.22:0","nonce":1013478817}]}]} 2024-02-10T07:44:27.885 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-10T07:44:27.885 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-10T07:44:27.885 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:44:28.128 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:28.374 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 podman[41018]: 2024-02-10 07:44:27.895288383 +0000 UTC m=+1.562404090 container init fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 podman[41018]: 2024-02-10 07:44:27.901247913 +0000 UTC m=+1.568363605 container start fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.927Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.927Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.927Z caller=main.go:570 level=info host_details="(Linux 5.14.0-419.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Feb 7 23:01:41 UTC 2024 x86_64 smithi027 (none))" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.927Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.927Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.928Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.929Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.931Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.931Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.933Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.933Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.452µs 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.933Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-10T07:44:28.375 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.933Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.933Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=23.27µs wal_replay_duration=223.537µs wbl_replay_duration=136ns total_replay_duration=267.734µs 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.934Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.934Z caller=main.go:1029 level=info msg="TSDB started" 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.934Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.953Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=18.389564ms db_storage=824ns remote_storage=1.624µs web_handler=265ns query_engine=809ns scrape=700.482µs scrape_sd=77.358µs notify=1.103µs notify_sd=2.368µs rules=17.382433ms tracing=7.364µs 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.953Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:44:27.953Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 bash[41018]: fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d 2024-02-10T07:44:28.376 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:44:27 smithi027 systemd[1]: Started Ceph prometheus.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setuser ceph since I am not root 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ignoring --setgroup ceph since I am not root 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:44:29.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-10T07:44:29.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: ignoring --setuser ceph since I am not root 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: ignoring --setgroup ceph since I am not root 2024-02-10T07:44:29.349 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-mgr[32257]: -- 172.21.15.81:0/2778036999 <== mon.1 v2:172.21.15.81:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564f742161a0 con 0x564f741ec800 2024-02-10T07:44:29.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: ignoring --setuser ceph since I am not root 2024-02-10T07:44:29.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: ignoring --setgroup ceph since I am not root 2024-02-10T07:44:29.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mgr[31332]: -- 172.21.15.27:0/3230396777 <== mon.0 v2:172.21.15.22:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564b903d61a0 con 0x564b903acc00 2024-02-10T07:44:29.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: pgmap v112: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 38 KiB/s rd, 2.8 KiB/s wr, 92 op/s 2024-02-10T07:44:29.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-10T07:44:29.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' 2024-02-10T07:44:29.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:29.661+0000 7f436c1d9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:44:29.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:29.759+0000 7f436c1d9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:44:29.995 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:44:29.996 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","created":"2024-02-10T07:40:28.951807+0000","modified":"2024-02-10T07:44:14.541339+0000","last_up_change":"2024-02-10T07:43:53.027732+0000","last_in_change":"2024-02-10T07:43:34.389966+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:43:53.910905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T07:44:05.953073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-10T07:44:07.815416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-10T07:44:09.772578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-10T07:44:11.842461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fe9ab54-a7ac-4c67-973d-3bd111153324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6803","nonce":3108645017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6805","nonce":3108645017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6809","nonce":3108645017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6807","nonce":3108645017}]},"public_addr":"172.21.15.22:6803/3108645017","cluster_addr":"172.21.15.22:6805/3108645017","heartbeat_back_addr":"172.21.15.22:6809/3108645017","heartbeat_front_addr":"172.21.15.22:6807/3108645017","state":["exists","up"]},{"osd":1,"uuid":"67a22425-d450-48ef-a0fe-c6310951cf06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6801","nonce":3302204668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6803","nonce":3302204668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6807","nonce":3302204668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6805","nonce":3302204668}]},"public_addr":"172.21.15.27:6801/3302204668","cluster_addr":"172.21.15.27:6803/3302204668","heartbeat_back_addr":"172.21.15.27:6807/3302204668","heartbeat_front_addr":"172.21.15.27:6805/3302204668","state":["exists","up"]},{"osd":2,"uuid":"36678171-675b-4433-b1f9-19cb96ed3d2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6801","nonce":1050443935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6803","nonce":1050443935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6807","nonce":1050443935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6805","nonce":1050443935}]},"public_addr":"172.21.15.81:6801/1050443935","cluster_addr":"172.21.15.81:6803/1050443935","heartbeat_back_addr":"172.21.15.81:6807/1050443935","heartbeat_front_addr":"172.21.15.81:6805/1050443935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:01.394570+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:26.927116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:51.212803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/2773249949":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4132953616":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4200425753":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/3221539624":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/3166432828":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/2917108429":"2024-02-11T07:40:56.450004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:44:30.051 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:29.611+0000 7f9596f18200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:44:30.051 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:29 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:29.704+0000 7f9596f18200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:44:30.052 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:29.641+0000 7fa820db7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-02-10T07:44:30.052 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:29 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:29.738+0000 7fa820db7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-10T07:44:30.312 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-mon[25704]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:44:30.312 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-mon[25704]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T07:44:30.312 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/104997270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:30.337 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-mon[30233]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:44:30.337 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-mon[30233]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T07:44:30.337 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/104997270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:30.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-mon[29844]: from='mgr.14152 172.21.15.22:0/3882800273' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-10T07:44:30.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-mon[29844]: mgrmap e17: y(active, since 3m), standbys: z, x 2024-02-10T07:44:30.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/104997270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:30.578 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:30.427+0000 7f436c1d9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:44:30.578 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:30.521+0000 7f436c1d9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:44:30.580 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-10T07:44:30.580 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd dump --format=json 2024-02-10T07:44:30.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:30.336+0000 7f9596f18200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:44:30.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:30.427+0000 7f9596f18200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:44:30.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:30.567+0000 7f9596f18200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:44:30.738 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:30.398+0000 7fa820db7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-10T07:44:30.738 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:30.494+0000 7fa820db7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-10T07:44:30.738 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:30.639+0000 7fa820db7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:44:30.738 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:30.737+0000 7fa820db7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:44:30.825 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:30.876 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:30.680+0000 7f436c1d9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-10T07:44:30.877 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:30.784+0000 7f436c1d9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:44:30.877 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:30 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:30.876+0000 7f436c1d9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:44:31.004 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:30.662+0000 7f9596f18200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-10T07:44:31.004 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:30 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:30.745+0000 7f9596f18200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:44:31.004 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:31 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:31.003+0000 7f9596f18200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:44:31.094 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:30 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:30.822+0000 7fa820db7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-10T07:44:31.294 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:31 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:31.113+0000 7f9596f18200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:31.295 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:31 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:31.199+0000 7f9596f18200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:44:31.295 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:31 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:31.294+0000 7f9596f18200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:44:31.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:31 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:31.094+0000 7fa820db7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:44:31.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:31 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:31.207+0000 7fa820db7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:31.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:31 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:31.295+0000 7fa820db7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:44:31.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:31 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:31.149+0000 7f436c1d9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-02-10T07:44:31.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:31 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:31.264+0000 7f436c1d9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:31.444 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:31 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:31.354+0000 7f436c1d9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-10T07:44:31.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:31 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:31.480+0000 7f9596f18200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:44:31.743 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:31 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:31.451+0000 7f436c1d9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:44:31.743 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:31 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:31.635+0000 7f436c1d9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:44:31.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:31 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:31.392+0000 7fa820db7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-10T07:44:31.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:31 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:31.573+0000 7fa820db7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-02-10T07:44:32.744 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:44:32.744 INFO:teuthology.orchestra.run.smithi022.stdout:{"epoch":30,"fsid":"80815fa4-c7e7-11ee-95b8-87774f69a715","created":"2024-02-10T07:40:28.951807+0000","modified":"2024-02-10T07:44:14.541339+0000","last_up_change":"2024-02-10T07:43:53.027732+0000","last_in_change":"2024-02-10T07:43:34.389966+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-02-10T07:43:53.910905+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3,"score_stable":3,"optimal_score":1,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-02-10T07:44:05.953073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.309999942779541,"score_stable":1.309999942779541,"optimal_score":1,"raw_score_acting":1.309999942779541,"raw_score_stable":1.309999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-02-10T07:44:07.815416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-02-10T07:44:09.772578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.meta","create_time":"2024-02-10T07:44:11.842461+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"30","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.2200000286102295,"score_stable":1.2200000286102295,"optimal_score":1,"raw_score_acting":1.2200000286102295,"raw_score_stable":1.2200000286102295,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1fe9ab54-a7ac-4c67-973d-3bd111153324","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6802","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6803","nonce":3108645017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6804","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6805","nonce":3108645017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6808","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6809","nonce":3108645017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.22:6806","nonce":3108645017},{"type":"v1","addr":"172.21.15.22:6807","nonce":3108645017}]},"public_addr":"172.21.15.22:6803/3108645017","cluster_addr":"172.21.15.22:6805/3108645017","heartbeat_back_addr":"172.21.15.22:6809/3108645017","heartbeat_front_addr":"172.21.15.22:6807/3108645017","state":["exists","up"]},{"osd":1,"uuid":"67a22425-d450-48ef-a0fe-c6310951cf06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6801","nonce":3302204668}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6803","nonce":3302204668}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6807","nonce":3302204668}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3302204668},{"type":"v1","addr":"172.21.15.27:6805","nonce":3302204668}]},"public_addr":"172.21.15.27:6801/3302204668","cluster_addr":"172.21.15.27:6803/3302204668","heartbeat_back_addr":"172.21.15.27:6807/3302204668","heartbeat_front_addr":"172.21.15.27:6805/3302204668","state":["exists","up"]},{"osd":2,"uuid":"36678171-675b-4433-b1f9-19cb96ed3d2f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6801","nonce":1050443935}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6803","nonce":1050443935}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6807","nonce":1050443935}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":1050443935},{"type":"v1","addr":"172.21.15.81:6805","nonce":1050443935}]},"public_addr":"172.21.15.81:6801/1050443935","cluster_addr":"172.21.15.81:6803/1050443935","heartbeat_back_addr":"172.21.15.81:6807/1050443935","heartbeat_front_addr":"172.21.15.81:6805/1050443935","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:01.394570+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:26.927116+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-02-10T07:43:51.212803+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.22:0/2773249949":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4132953616":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/3297599050":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/4200425753":"2024-02-11T07:41:23.562856+0000","172.21.15.22:0/3221539624":"2024-02-11T07:41:23.562856+0000","172.21.15.22:6801/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:6800/767740731":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/3166432828":"2024-02-11T07:40:56.450004+0000","172.21.15.22:0/2917108429":"2024-02-11T07:40:56.450004+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-10T07:44:32.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:32 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:32.435+0000 7f9596f18200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:44:32.794 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:32 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:32.567+0000 7f9596f18200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:44:32.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:32 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2735008626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:32.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:32 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:32.582+0000 7fa820db7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:44:32.874 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:32 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:32.717+0000 7fa820db7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:44:32.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:32 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2735008626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:32.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:32.642+0000 7f436c1d9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-10T07:44:32.944 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:32.780+0000 7f436c1d9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-10T07:44:33.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:32 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2735008626' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-10T07:44:33.097 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:32 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:32.911+0000 7f9596f18200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:44:33.097 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:33 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:33.001+0000 7f9596f18200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:44:33.215 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:33 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:33.122+0000 7f436c1d9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:44:33.215 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:33 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:33.215+0000 7f436c1d9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:44:33.357 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:33 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:33.062+0000 7fa820db7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-10T07:44:33.357 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:33 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:33.155+0000 7fa820db7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-10T07:44:33.392 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:33 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:33.200+0000 7f9596f18200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:33.392 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:33 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:33.391+0000 7f9596f18200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:44:33.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-10T07:44:33.444 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-10T07:44:33.445 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-10T07:44:33.470 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:33 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:33.417+0000 7f436c1d9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:33.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:33 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:33.357+0000 7fa820db7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:33.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:33 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:33.548+0000 7fa820db7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:44:33.720 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:33.721 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:33.722 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:33.846 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:33 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:33.623+0000 7f436c1d9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-10T07:44:33.955 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:33 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:33.690+0000 7f9596f18200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:34.119 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:33 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:33.843+0000 7fa820db7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:34.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:33 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:33.958+0000 7f436c1d9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-10T07:44:34.267 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:33 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:33.954+0000 7f9596f18200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:34.268 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:34 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:34.194+0000 7f9596f18200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:44:34.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:34 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:34.118+0000 7fa820db7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:34.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:34 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:34.360+0000 7fa820db7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:44:34.503 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:34 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:34.232+0000 7f436c1d9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-10T07:44:34.848 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:34 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:34.502+0000 7f436c1d9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-10T07:44:35.347 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:34 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:34.988+0000 7f9596f18200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:44:35.624 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:35.190+0000 7fa820db7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:44:35.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:35.373+0000 7f436c1d9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-10T07:44:35.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:35.551+0000 7f9596f18200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:44:35.847 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:35.649+0000 7f9596f18200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: Standby manager daemon z restarted 2024-02-10T07:44:35.992 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: Standby manager daemon z started 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:44:35.993 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:36.107 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:35.781+0000 7fa820db7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:44:36.107 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:35.882+0000 7fa820db7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: Standby manager daemon z restarted 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: Standby manager daemon z started 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:44:36.108 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:35 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:36.320 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:35 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:35.991+0000 7f436c1d9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-10T07:44:36.320 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:36.094+0000 7f436c1d9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-10T07:44:36.320 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:36.320+0000 7f436c1d9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:44:36.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: Standby manager daemon z restarted 2024-02-10T07:44:36.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: Standby manager daemon z started 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.81:0/2979546924' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:35.863+0000 7f9596f18200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: [10/Feb/2024:07:44:35] ENGINE Bus STARTING 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: CherryPy Checker: 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: The Application mounted at '' has an empty config. 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: [10/Feb/2024:07:44:35] ENGINE Serving on http://:::9283 2024-02-10T07:44:36.348 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:44:35 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z[32201]: [10/Feb/2024:07:44:35] ENGINE Bus STARTED 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:36.104+0000 7fa820db7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: [10/Feb/2024:07:44:36] ENGINE Bus STARTING 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: CherryPy Checker: 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: The Application mounted at '' has an empty config. 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: [10/Feb/2024:07:44:36] ENGINE Serving on http://:::9283 2024-02-10T07:44:36.356 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x[31309]: [10/Feb/2024:07:44:36] ENGINE Bus STARTED 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:36] ENGINE Bus STARTING 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: CherryPy Checker: 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: The Application mounted at '' has an empty config. 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:36] ENGINE Serving on http://:::9283 2024-02-10T07:44:36.600 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:36] ENGINE Bus STARTED 2024-02-10T07:44:37.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T07:44:37.137 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: Standby manager daemon x restarted 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: Standby manager daemon x started 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: Active manager daemon y restarted 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: Activating manager daemon y 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: mgrmap e19: y(active, starting, since 0.00989817s), standbys: z, x 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:44:37.138 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: Manager daemon y is now available 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:44:37.139 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:36 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: Standby manager daemon x restarted 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: Standby manager daemon x started 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:37.305 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: Active manager daemon y restarted 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: Activating manager daemon y 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: mgrmap e19: y(active, starting, since 0.00989817s), standbys: z, x 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:44:37.306 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: Manager daemon y is now available 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:44:37.307 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:36 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: mgrmap e18: y(active, since 3m), standbys: x, z 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: Standby manager daemon x restarted 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: Standby manager daemon x started 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-10T07:44:37.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.? 172.21.15.27:0/887333268' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: Active manager daemon y restarted 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: Activating manager daemon y 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: osdmap e31: 3 total, 3 up, 3 in 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: mgrmap e19: y(active, starting, since 0.00989817s), standbys: z, x 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-10T07:44:37.349 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: Manager daemon y is now available 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-10T07:44:37.350 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:36 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-10T07:44:38.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:37 smithi022 ceph-mon[25704]: [10/Feb/2024:07:44:37] ENGINE Bus STARTING 2024-02-10T07:44:38.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:37 smithi022 ceph-mon[25704]: mgrmap e20: y(active, since 1.01499s), standbys: z, x 2024-02-10T07:44:38.338 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:37 smithi027 ceph-mon[29844]: [10/Feb/2024:07:44:37] ENGINE Bus STARTING 2024-02-10T07:44:38.338 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:37 smithi027 ceph-mon[29844]: mgrmap e20: y(active, since 1.01499s), standbys: z, x 2024-02-10T07:44:38.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:37 smithi081 ceph-mon[30233]: [10/Feb/2024:07:44:37] ENGINE Bus STARTING 2024-02-10T07:44:38.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:37 smithi081 ceph-mon[30233]: mgrmap e20: y(active, since 1.01499s), standbys: z, x 2024-02-10T07:44:38.605 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:38 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:44:38] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:38.637 INFO:teuthology.orchestra.run.smithi022.stdout:77309411340 2024-02-10T07:44:38.637 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-10T07:44:39.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: [10/Feb/2024:07:44:37] ENGINE Serving on https://172.21.15.22:7150 2024-02-10T07:44:39.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: [10/Feb/2024:07:44:37] ENGINE Serving on http://172.21.15.22:8765 2024-02-10T07:44:39.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: [10/Feb/2024:07:44:37] ENGINE Bus STARTED 2024-02-10T07:44:39.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:39.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.195 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:39 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.311 INFO:teuthology.orchestra.run.smithi022.stdout:34359738390 2024-02-10T07:44:39.312 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: [10/Feb/2024:07:44:37] ENGINE Serving on https://172.21.15.22:7150 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: [10/Feb/2024:07:44:37] ENGINE Serving on http://172.21.15.22:8765 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: [10/Feb/2024:07:44:37] ENGINE Bus STARTED 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:39 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: [10/Feb/2024:07:44:37] ENGINE Serving on https://172.21.15.22:7150 2024-02-10T07:44:39.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: [10/Feb/2024:07:44:37] ENGINE Serving on http://172.21.15.22:8765 2024-02-10T07:44:39.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: [10/Feb/2024:07:44:37] ENGINE Bus STARTED 2024-02-10T07:44:39.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: pgmap v3: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:39.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:39.376 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:39 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:40.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:40.348 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:40 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:40.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:40 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: pgmap v4: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: mgrmap e21: y(active, since 3s), standbys: z, x 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:40.408 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:40 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:40.434 INFO:teuthology.orchestra.run.smithi022.stdout:55834574864 2024-02-10T07:44:40.434 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-10T07:44:40.530 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:40.878 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:41.306 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:41 smithi022 ceph-mon[25704]: Adjusting osd_memory_target on smithi027 to 16064M 2024-02-10T07:44:41.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:41 smithi027 ceph-mon[29844]: Adjusting osd_memory_target on smithi027 to 16064M 2024-02-10T07:44:41.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:41 smithi081 ceph-mon[30233]: Adjusting osd_memory_target on smithi027 to 16064M 2024-02-10T07:44:42.154 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:42.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:42 smithi022 ceph-mon[25704]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-10T07:44:42.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:42 smithi022 ceph-mon[25704]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-10T07:44:42.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:42 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:42.252 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:42 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:42.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:42 smithi027 ceph-mon[29844]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-10T07:44:42.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:42 smithi027 ceph-mon[29844]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-10T07:44:42.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:42 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:42.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:42 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:42.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:42 smithi081 ceph-mon[30233]: pgmap v5: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 69 KiB/s rd, 0 B/s wr, 38 op/s 2024-02-10T07:44:42.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:42 smithi081 ceph-mon[30233]: mgrmap e22: y(active, since 4s), standbys: z, x 2024-02-10T07:44:42.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:42 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:42.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:42 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:44.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:44 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:44.382 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:44.383 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:44 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:44.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: pgmap v6: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 41 KiB/s rd, 0 B/s wr, 23 op/s 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-10T07:44:44.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:44 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-10T07:44:44.859 INFO:teuthology.orchestra.run.smithi022.stdout:77309411340 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Adjusting osd_memory_target on smithi022 to 16069M 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.309 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.310 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:45 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3775701980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:44:45.337 INFO:teuthology.orchestra.run.smithi022.stdout:55834574865 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Adjusting osd_memory_target on smithi022 to 16069M 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:45 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/3775701980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:44:45.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Adjusting osd_memory_target on smithi022 to 16069M 2024-02-10T07:44:45.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi022:/etc/ceph/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi027:/etc/ceph/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi081:/etc/ceph/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.conf 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:45.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:45 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/3775701980' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-10T07:44:45.684 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411340 got 77309411340 for osd.2 2024-02-10T07:44:45.685 DEBUG:teuthology.parallel:result is None 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: Deploying daemon alertmanager.a on smithi022 2024-02-10T07:44:46.272 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:46 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/3142793622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: Deploying daemon alertmanager.a on smithi022 2024-02-10T07:44:46.375 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:46 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/3142793622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:44:46.591 INFO:teuthology.orchestra.run.smithi022.stdout:34359738391 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi022:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi081:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi027:/etc/ceph/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: pgmap v7: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Updating smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/config/ceph.client.admin.keyring 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: Deploying daemon alertmanager.a on smithi022 2024-02-10T07:44:46.598 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:46 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/3142793622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-10T07:44:46.850 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574864 got 55834574865 for osd.1 2024-02-10T07:44:46.850 DEBUG:teuthology.parallel:result is None 2024-02-10T07:44:47.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:47 smithi027 ceph-mon[29844]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T07:44:47.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:47 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/4132546496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:44:47.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:47 smithi022 ceph-mon[25704]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T07:44:47.487 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:47 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/4132546496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:44:47.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:47 smithi081 ceph-mon[30233]: pgmap v8: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-02-10T07:44:47.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:47 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/4132546496' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-10T07:44:47.766 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738390 got 34359738391 for osd.0 2024-02-10T07:44:47.766 DEBUG:teuthology.parallel:result is None 2024-02-10T07:44:47.766 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-10T07:44:47.766 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T07:44:48.084 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:48.613 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:48 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:44:48] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:49.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:49 smithi022 ceph-mon[25704]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:44:49.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:49 smithi081 ceph-mon[30233]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:44:49.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:49 smithi027 ceph-mon[29844]: pgmap v9: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-02-10T07:44:51.267 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:44:51.270 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-02-10T07:44:51.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:51 smithi022 ceph-mon[25704]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T07:44:51.647 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:51 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:51 smithi081 ceph-mon[30233]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T07:44:51.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:51 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:51 smithi027 ceph-mon[29844]: pgmap v10: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-02-10T07:44:51.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:51 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:44:52.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:52 smithi081 ceph-mon[30233]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:52.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:52 smithi027 ceph-mon[29844]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:52.887 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:52 smithi022 ceph-mon[25704]: from='client.14538 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:52.920 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-02-10T07:44:50.351246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96620,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126548,"statfs":{"total":287972524032,"available":287873585152,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":95743312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002888"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341110+0000","last_change":"2024-02-10T07:44:13.557527+0000","last_active":"2024-02-10T07:44:36.341110+0000","last_peered":"2024-02-10T07:44:36.341110+0000","last_clean":"2024-02-10T07:44:36.341110+0000","last_became_active":"2024-02-10T07:44:13.556947+0000","last_became_peered":"2024-02-10T07:44:13.556947+0000","last_unstale":"2024-02-10T07:44:36.341110+0000","last_undegraded":"2024-02-10T07:44:36.341110+0000","last_fullsized":"2024-02-10T07:44:36.341110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:13:46.282106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340176+0000","last_change":"2024-02-10T07:44:11.538312+0000","last_active":"2024-02-10T07:44:36.340176+0000","last_peered":"2024-02-10T07:44:36.340176+0000","last_clean":"2024-02-10T07:44:36.340176+0000","last_became_active":"2024-02-10T07:44:11.537998+0000","last_became_peered":"2024-02-10T07:44:11.537998+0000","last_unstale":"2024-02-10T07:44:36.340176+0000","last_undegraded":"2024-02-10T07:44:36.340176+0000","last_fullsized":"2024-02-10T07:44:36.340176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:13:17.152364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342024+0000","last_change":"2024-02-10T07:44:07.511088+0000","last_active":"2024-02-10T07:44:36.342024+0000","last_peered":"2024-02-10T07:44:36.342024+0000","last_clean":"2024-02-10T07:44:36.342024+0000","last_became_active":"2024-02-10T07:44:07.510385+0000","last_became_peered":"2024-02-10T07:44:07.510385+0000","last_unstale":"2024-02-10T07:44:36.342024+0000","last_undegraded":"2024-02-10T07:44:36.342024+0000","last_fullsized":"2024-02-10T07:44:36.342024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:24:28.408798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341701+0000","last_change":"2024-02-10T07:44:09.527197+0000","last_active":"2024-02-10T07:44:36.341701+0000","last_peered":"2024-02-10T07:44:36.341701+0000","last_clean":"2024-02-10T07:44:36.341701+0000","last_became_active":"2024-02-10T07:44:09.526598+0000","last_became_peered":"2024-02-10T07:44:09.526598+0000","last_unstale":"2024-02-10T07:44:36.341701+0000","last_undegraded":"2024-02-10T07:44:36.341701+0000","last_fullsized":"2024-02-10T07:44:36.341701+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:43:11.136118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339392+0000","last_change":"2024-02-10T07:44:13.555219+0000","last_active":"2024-02-10T07:44:36.339392+0000","last_peered":"2024-02-10T07:44:36.339392+0000","last_clean":"2024-02-10T07:44:36.339392+0000","last_became_active":"2024-02-10T07:44:13.553853+0000","last_became_peered":"2024-02-10T07:44:13.553853+0000","last_unstale":"2024-02-10T07:44:36.339392+0000","last_undegraded":"2024-02-10T07:44:36.339392+0000","last_fullsized":"2024-02-10T07:44:36.339392+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:16:53.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340468+0000","last_change":"2024-02-10T07:44:11.543384+0000","last_active":"2024-02-10T07:44:36.340468+0000","last_peered":"2024-02-10T07:44:36.340468+0000","last_clean":"2024-02-10T07:44:36.340468+0000","last_became_active":"2024-02-10T07:44:11.542792+0000","last_became_peered":"2024-02-10T07:44:11.542792+0000","last_unstale":"2024-02-10T07:44:36.340468+0000","last_undegraded":"2024-02-10T07:44:36.340468+0000","last_fullsized":"2024-02-10T07:44:36.340468+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:11:32.529921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341991+0000","last_change":"2024-02-10T07:44:07.510076+0000","last_active":"2024-02-10T07:44:36.341991+0000","last_peered":"2024-02-10T07:44:36.341991+0000","last_clean":"2024-02-10T07:44:36.341991+0000","last_became_active":"2024-02-10T07:44:07.509467+0000","last_became_peered":"2024-02-10T07:44:07.509467+0000","last_unstale":"2024-02-10T07:44:36.341991+0000","last_undegraded":"2024-02-10T07:44:36.341991+0000","last_fullsized":"2024-02-10T07:44:36.341991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:46.327660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341624+0000","last_change":"2024-02-10T07:44:09.526396+0000","last_active":"2024-02-10T07:44:36.341624+0000","last_peered":"2024-02-10T07:44:36.341624+0000","last_clean":"2024-02-10T07:44:36.341624+0000","last_became_active":"2024-02-10T07:44:09.526081+0000","last_became_peered":"2024-02-10T07:44:09.526081+0000","last_unstale":"2024-02-10T07:44:36.341624+0000","last_undegraded":"2024-02-10T07:44:36.341624+0000","last_fullsized":"2024-02-10T07:44:36.341624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:02:58.093487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340238+0000","last_change":"2024-02-10T07:44:13.554626+0000","last_active":"2024-02-10T07:44:36.340238+0000","last_peered":"2024-02-10T07:44:36.340238+0000","last_clean":"2024-02-10T07:44:36.340238+0000","last_became_active":"2024-02-10T07:44:13.554344+0000","last_became_peered":"2024-02-10T07:44:13.554344+0000","last_unstale":"2024-02-10T07:44:36.340238+0000","last_undegraded":"2024-02-10T07:44:36.340238+0000","last_fullsized":"2024-02-10T07:44:36.340238+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:11:50.006023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341506+0000","last_change":"2024-02-10T07:44:11.541610+0000","last_active":"2024-02-10T07:44:36.341506+0000","last_peered":"2024-02-10T07:44:36.341506+0000","last_clean":"2024-02-10T07:44:36.341506+0000","last_became_active":"2024-02-10T07:44:11.541270+0000","last_became_peered":"2024-02-10T07:44:11.541270+0000","last_unstale":"2024-02-10T07:44:36.341506+0000","last_undegraded":"2024-02-10T07:44:36.341506+0000","last_fullsized":"2024-02-10T07:44:36.341506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:57.155483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343371+0000","last_change":"2024-02-10T07:44:07.513280+0000","last_active":"2024-02-10T07:44:36.343371+0000","last_peered":"2024-02-10T07:44:36.343371+0000","last_clean":"2024-02-10T07:44:36.343371+0000","last_became_active":"2024-02-10T07:44:07.512394+0000","last_became_peered":"2024-02-10T07:44:07.512394+0000","last_unstale":"2024-02-10T07:44:36.343371+0000","last_undegraded":"2024-02-10T07:44:36.343371+0000","last_fullsized":"2024-02-10T07:44:36.343371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:21.926453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343033+0000","last_change":"2024-02-10T07:44:09.522685+0000","last_active":"2024-02-10T07:44:36.343033+0000","last_peered":"2024-02-10T07:44:36.343033+0000","last_clean":"2024-02-10T07:44:36.343033+0000","last_became_active":"2024-02-10T07:44:09.522010+0000","last_became_peered":"2024-02-10T07:44:09.522010+0000","last_unstale":"2024-02-10T07:44:36.343033+0000","last_undegraded":"2024-02-10T07:44:36.343033+0000","last_fullsized":"2024-02-10T07:44:36.343033+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:08:42.582421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340472+0000","last_change":"2024-02-10T07:44:13.554941+0000","last_active":"2024-02-10T07:44:36.340472+0000","last_peered":"2024-02-10T07:44:36.340472+0000","last_clean":"2024-02-10T07:44:36.340472+0000","last_became_active":"2024-02-10T07:44:13.554387+0000","last_became_peered":"2024-02-10T07:44:13.554387+0000","last_unstale":"2024-02-10T07:44:36.340472+0000","last_undegraded":"2024-02-10T07:44:36.340472+0000","last_fullsized":"2024-02-10T07:44:36.340472+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:37:28.332666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340582+0000","last_change":"2024-02-10T07:44:11.542774+0000","last_active":"2024-02-10T07:44:36.340582+0000","last_peered":"2024-02-10T07:44:36.340582+0000","last_clean":"2024-02-10T07:44:36.340582+0000","last_became_active":"2024-02-10T07:44:11.541837+0000","last_became_peered":"2024-02-10T07:44:11.541837+0000","last_unstale":"2024-02-10T07:44:36.340582+0000","last_undegraded":"2024-02-10T07:44:36.340582+0000","last_fullsized":"2024-02-10T07:44:36.340582+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:17:01.022750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341816+0000","last_change":"2024-02-10T07:44:07.510728+0000","last_active":"2024-02-10T07:44:36.341816+0000","last_peered":"2024-02-10T07:44:36.341816+0000","last_clean":"2024-02-10T07:44:36.341816+0000","last_became_active":"2024-02-10T07:44:07.510470+0000","last_became_peered":"2024-02-10T07:44:07.510470+0000","last_unstale":"2024-02-10T07:44:36.341816+0000","last_undegraded":"2024-02-10T07:44:36.341816+0000","last_fullsized":"2024-02-10T07:44:36.341816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:19:30.919065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343326+0000","last_change":"2024-02-10T07:44:09.522627+0000","last_active":"2024-02-10T07:44:36.343326+0000","last_peered":"2024-02-10T07:44:36.343326+0000","last_clean":"2024-02-10T07:44:36.343326+0000","last_became_active":"2024-02-10T07:44:09.522043+0000","last_became_peered":"2024-02-10T07:44:09.522043+0000","last_unstale":"2024-02-10T07:44:36.343326+0000","last_undegraded":"2024-02-10T07:44:36.343326+0000","last_fullsized":"2024-02-10T07:44:36.343326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:07:25.598568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341866+0000","last_change":"2024-02-10T07:44:13.556884+0000","last_active":"2024-02-10T07:44:36.341866+0000","last_peered":"2024-02-10T07:44:36.341866+0000","last_clean":"2024-02-10T07:44:36.341866+0000","last_became_active":"2024-02-10T07:44:13.555485+0000","last_became_peered":"2024-02-10T07:44:13.555485+0000","last_unstale":"2024-02-10T07:44:36.341866+0000","last_undegraded":"2024-02-10T07:44:36.341866+0000","last_fullsized":"2024-02-10T07:44:36.341866+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:33:48.427105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342878+0000","last_change":"2024-02-10T07:44:11.541138+0000","last_active":"2024-02-10T07:44:36.342878+0000","last_peered":"2024-02-10T07:44:36.342878+0000","last_clean":"2024-02-10T07:44:36.342878+0000","last_became_active":"2024-02-10T07:44:11.540840+0000","last_became_peered":"2024-02-10T07:44:11.540840+0000","last_unstale":"2024-02-10T07:44:36.342878+0000","last_undegraded":"2024-02-10T07:44:36.342878+0000","last_fullsized":"2024-02-10T07:44:36.342878+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:46:51.081334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341796+0000","last_change":"2024-02-10T07:44:07.509381+0000","last_active":"2024-02-10T07:44:36.341796+0000","last_peered":"2024-02-10T07:44:36.341796+0000","last_clean":"2024-02-10T07:44:36.341796+0000","last_became_active":"2024-02-10T07:44:07.509073+0000","last_became_peered":"2024-02-10T07:44:07.509073+0000","last_unstale":"2024-02-10T07:44:36.341796+0000","last_undegraded":"2024-02-10T07:44:36.341796+0000","last_fullsized":"2024-02-10T07:44:36.341796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:22:33.069618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340982+0000","last_change":"2024-02-10T07:44:09.523039+0000","last_active":"2024-02-10T07:44:36.340982+0000","last_peered":"2024-02-10T07:44:36.340982+0000","last_clean":"2024-02-10T07:44:36.340982+0000","last_became_active":"2024-02-10T07:44:09.522401+0000","last_became_peered":"2024-02-10T07:44:09.522401+0000","last_unstale":"2024-02-10T07:44:36.340982+0000","last_undegraded":"2024-02-10T07:44:36.340982+0000","last_fullsized":"2024-02-10T07:44:36.340982+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:11:08.947906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340799+0000","last_change":"2024-02-10T07:44:13.560016+0000","last_active":"2024-02-10T07:44:36.340799+0000","last_peered":"2024-02-10T07:44:36.340799+0000","last_clean":"2024-02-10T07:44:36.340799+0000","last_became_active":"2024-02-10T07:44:13.559681+0000","last_became_peered":"2024-02-10T07:44:13.559681+0000","last_unstale":"2024-02-10T07:44:36.340799+0000","last_undegraded":"2024-02-10T07:44:36.340799+0000","last_fullsized":"2024-02-10T07:44:36.340799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:46:38.015414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341594+0000","last_change":"2024-02-10T07:44:11.537824+0000","last_active":"2024-02-10T07:44:36.341594+0000","last_peered":"2024-02-10T07:44:36.341594+0000","last_clean":"2024-02-10T07:44:36.341594+0000","last_became_active":"2024-02-10T07:44:11.537544+0000","last_became_peered":"2024-02-10T07:44:11.537544+0000","last_unstale":"2024-02-10T07:44:36.341594+0000","last_undegraded":"2024-02-10T07:44:36.341594+0000","last_fullsized":"2024-02-10T07:44:36.341594+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:12:03.769889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340851+0000","last_change":"2024-02-10T07:44:07.512551+0000","last_active":"2024-02-10T07:44:36.340851+0000","last_peered":"2024-02-10T07:44:36.340851+0000","last_clean":"2024-02-10T07:44:36.340851+0000","last_became_active":"2024-02-10T07:44:07.512253+0000","last_became_peered":"2024-02-10T07:44:07.512253+0000","last_unstale":"2024-02-10T07:44:36.340851+0000","last_undegraded":"2024-02-10T07:44:36.340851+0000","last_fullsized":"2024-02-10T07:44:36.340851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:10:02.236421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341509+0000","last_change":"2024-02-10T07:44:09.522161+0000","last_active":"2024-02-10T07:44:36.341509+0000","last_peered":"2024-02-10T07:44:36.341509+0000","last_clean":"2024-02-10T07:44:36.341509+0000","last_became_active":"2024-02-10T07:44:09.521867+0000","last_became_peered":"2024-02-10T07:44:09.521867+0000","last_unstale":"2024-02-10T07:44:36.341509+0000","last_undegraded":"2024-02-10T07:44:36.341509+0000","last_fullsized":"2024-02-10T07:44:36.341509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:38:38.883409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341121+0000","last_change":"2024-02-10T07:44:13.555482+0000","last_active":"2024-02-10T07:44:36.341121+0000","last_peered":"2024-02-10T07:44:36.341121+0000","last_clean":"2024-02-10T07:44:36.341121+0000","last_became_active":"2024-02-10T07:44:13.553979+0000","last_became_peered":"2024-02-10T07:44:13.553979+0000","last_unstale":"2024-02-10T07:44:36.341121+0000","last_undegraded":"2024-02-10T07:44:36.341121+0000","last_fullsized":"2024-02-10T07:44:36.341121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:24:48.122612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562276+0000","last_change":"2024-02-10T07:44:11.542004+0000","last_active":"2024-02-10T07:44:37.562276+0000","last_peered":"2024-02-10T07:44:37.562276+0000","last_clean":"2024-02-10T07:44:37.562276+0000","last_became_active":"2024-02-10T07:44:11.541147+0000","last_became_peered":"2024-02-10T07:44:11.541147+0000","last_unstale":"2024-02-10T07:44:37.562276+0000","last_undegraded":"2024-02-10T07:44:37.562276+0000","last_fullsized":"2024-02-10T07:44:37.562276+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:24:45.218856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339535+0000","last_change":"2024-02-10T07:44:07.511548+0000","last_active":"2024-02-10T07:44:36.339535+0000","last_peered":"2024-02-10T07:44:36.339535+0000","last_clean":"2024-02-10T07:44:36.339535+0000","last_became_active":"2024-02-10T07:44:07.510487+0000","last_became_peered":"2024-02-10T07:44:07.510487+0000","last_unstale":"2024-02-10T07:44:36.339535+0000","last_undegraded":"2024-02-10T07:44:36.339535+0000","last_fullsized":"2024-02-10T07:44:36.339535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:27:17.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338674+0000","last_change":"2024-02-10T07:44:09.527066+0000","last_active":"2024-02-10T07:44:36.338674+0000","last_peered":"2024-02-10T07:44:36.338674+0000","last_clean":"2024-02-10T07:44:36.338674+0000","last_became_active":"2024-02-10T07:44:09.526764+0000","last_became_peered":"2024-02-10T07:44:09.526764+0000","last_unstale":"2024-02-10T07:44:36.338674+0000","last_undegraded":"2024-02-10T07:44:36.338674+0000","last_fullsized":"2024-02-10T07:44:36.338674+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:45:11.605353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341411+0000","last_change":"2024-02-10T07:44:13.556463+0000","last_active":"2024-02-10T07:44:36.341411+0000","last_peered":"2024-02-10T07:44:36.341411+0000","last_clean":"2024-02-10T07:44:36.341411+0000","last_became_active":"2024-02-10T07:44:13.555226+0000","last_became_peered":"2024-02-10T07:44:13.555226+0000","last_unstale":"2024-02-10T07:44:36.341411+0000","last_undegraded":"2024-02-10T07:44:36.341411+0000","last_fullsized":"2024-02-10T07:44:36.341411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:08:32.539880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341284+0000","last_change":"2024-02-10T07:44:11.537841+0000","last_active":"2024-02-10T07:44:36.341284+0000","last_peered":"2024-02-10T07:44:36.341284+0000","last_clean":"2024-02-10T07:44:36.341284+0000","last_became_active":"2024-02-10T07:44:11.536933+0000","last_became_peered":"2024-02-10T07:44:11.536933+0000","last_unstale":"2024-02-10T07:44:36.341284+0000","last_undegraded":"2024-02-10T07:44:36.341284+0000","last_fullsized":"2024-02-10T07:44:36.341284+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:01:36.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340432+0000","last_change":"2024-02-10T07:44:07.513225+0000","last_active":"2024-02-10T07:44:36.340432+0000","last_peered":"2024-02-10T07:44:36.340432+0000","last_clean":"2024-02-10T07:44:36.340432+0000","last_became_active":"2024-02-10T07:44:07.512722+0000","last_became_peered":"2024-02-10T07:44:07.512722+0000","last_unstale":"2024-02-10T07:44:36.340432+0000","last_undegraded":"2024-02-10T07:44:36.340432+0000","last_fullsized":"2024-02-10T07:44:36.340432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:35:33.410717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339092+0000","last_change":"2024-02-10T07:44:09.526189+0000","last_active":"2024-02-10T07:44:36.339092+0000","last_peered":"2024-02-10T07:44:36.339092+0000","last_clean":"2024-02-10T07:44:36.339092+0000","last_became_active":"2024-02-10T07:44:09.525787+0000","last_became_peered":"2024-02-10T07:44:09.525787+0000","last_unstale":"2024-02-10T07:44:36.339092+0000","last_undegraded":"2024-02-10T07:44:36.339092+0000","last_fullsized":"2024-02-10T07:44:36.339092+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:35:04.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341272+0000","last_change":"2024-02-10T07:44:13.556920+0000","last_active":"2024-02-10T07:44:36.341272+0000","last_peered":"2024-02-10T07:44:36.341272+0000","last_clean":"2024-02-10T07:44:36.341272+0000","last_became_active":"2024-02-10T07:44:13.556458+0000","last_became_peered":"2024-02-10T07:44:13.556458+0000","last_unstale":"2024-02-10T07:44:36.341272+0000","last_undegraded":"2024-02-10T07:44:36.341272+0000","last_fullsized":"2024-02-10T07:44:36.341272+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:09:02.836089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338923+0000","last_change":"2024-02-10T07:44:11.535402+0000","last_active":"2024-02-10T07:44:36.338923+0000","last_peered":"2024-02-10T07:44:36.338923+0000","last_clean":"2024-02-10T07:44:36.338923+0000","last_became_active":"2024-02-10T07:44:11.534703+0000","last_became_peered":"2024-02-10T07:44:11.534703+0000","last_unstale":"2024-02-10T07:44:36.338923+0000","last_undegraded":"2024-02-10T07:44:36.338923+0000","last_fullsized":"2024-02-10T07:44:36.338923+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:36:13.180093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342959+0000","last_change":"2024-02-10T07:44:07.514066+0000","last_active":"2024-02-10T07:44:36.342959+0000","last_peered":"2024-02-10T07:44:36.342959+0000","last_clean":"2024-02-10T07:44:36.342959+0000","last_became_active":"2024-02-10T07:44:07.513754+0000","last_became_peered":"2024-02-10T07:44:07.513754+0000","last_unstale":"2024-02-10T07:44:36.342959+0000","last_undegraded":"2024-02-10T07:44:36.342959+0000","last_fullsized":"2024-02-10T07:44:36.342959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:06:04.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343543+0000","last_change":"2024-02-10T07:44:09.522767+0000","last_active":"2024-02-10T07:44:36.343543+0000","last_peered":"2024-02-10T07:44:36.343543+0000","last_clean":"2024-02-10T07:44:36.343543+0000","last_became_active":"2024-02-10T07:44:09.522208+0000","last_became_peered":"2024-02-10T07:44:09.522208+0000","last_unstale":"2024-02-10T07:44:36.343543+0000","last_undegraded":"2024-02-10T07:44:36.343543+0000","last_fullsized":"2024-02-10T07:44:36.343543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:43:13.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340999+0000","last_change":"2024-02-10T07:44:13.555142+0000","last_active":"2024-02-10T07:44:36.340999+0000","last_peered":"2024-02-10T07:44:36.340999+0000","last_clean":"2024-02-10T07:44:36.340999+0000","last_became_active":"2024-02-10T07:44:13.554155+0000","last_became_peered":"2024-02-10T07:44:13.554155+0000","last_unstale":"2024-02-10T07:44:36.340999+0000","last_undegraded":"2024-02-10T07:44:36.340999+0000","last_fullsized":"2024-02-10T07:44:36.340999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:15:55.124063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339024+0000","last_change":"2024-02-10T07:44:11.539614+0000","last_active":"2024-02-10T07:44:36.339024+0000","last_peered":"2024-02-10T07:44:36.339024+0000","last_clean":"2024-02-10T07:44:36.339024+0000","last_became_active":"2024-02-10T07:44:11.538208+0000","last_became_peered":"2024-02-10T07:44:11.538208+0000","last_unstale":"2024-02-10T07:44:36.339024+0000","last_undegraded":"2024-02-10T07:44:36.339024+0000","last_fullsized":"2024-02-10T07:44:36.339024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:23:06.358992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343392+0000","last_change":"2024-02-10T07:44:07.513398+0000","last_active":"2024-02-10T07:44:36.343392+0000","last_peered":"2024-02-10T07:44:36.343392+0000","last_clean":"2024-02-10T07:44:36.343392+0000","last_became_active":"2024-02-10T07:44:07.513127+0000","last_became_peered":"2024-02-10T07:44:07.513127+0000","last_unstale":"2024-02-10T07:44:36.343392+0000","last_undegraded":"2024-02-10T07:44:36.343392+0000","last_fullsized":"2024-02-10T07:44:36.343392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:59:00.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341809+0000","last_change":"2024-02-10T07:44:09.527791+0000","last_active":"2024-02-10T07:44:36.341809+0000","last_peered":"2024-02-10T07:44:36.341809+0000","last_clean":"2024-02-10T07:44:36.341809+0000","last_became_active":"2024-02-10T07:44:09.527510+0000","last_became_peered":"2024-02-10T07:44:09.527510+0000","last_unstale":"2024-02-10T07:44:36.341809+0000","last_undegraded":"2024-02-10T07:44:36.341809+0000","last_fullsized":"2024-02-10T07:44:36.341809+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:26:21.793062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340640+0000","last_change":"2024-02-10T07:44:13.554681+0000","last_active":"2024-02-10T07:44:36.340640+0000","last_peered":"2024-02-10T07:44:36.340640+0000","last_clean":"2024-02-10T07:44:36.340640+0000","last_became_active":"2024-02-10T07:44:13.554408+0000","last_became_peered":"2024-02-10T07:44:13.554408+0000","last_unstale":"2024-02-10T07:44:36.340640+0000","last_undegraded":"2024-02-10T07:44:36.340640+0000","last_fullsized":"2024-02-10T07:44:36.340640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:40:19.737670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341679+0000","last_change":"2024-02-10T07:44:11.543485+0000","last_active":"2024-02-10T07:44:36.341679+0000","last_peered":"2024-02-10T07:44:36.341679+0000","last_clean":"2024-02-10T07:44:36.341679+0000","last_became_active":"2024-02-10T07:44:11.543132+0000","last_became_peered":"2024-02-10T07:44:11.543132+0000","last_unstale":"2024-02-10T07:44:36.341679+0000","last_undegraded":"2024-02-10T07:44:36.341679+0000","last_fullsized":"2024-02-10T07:44:36.341679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:16:12.458455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343267+0000","last_change":"2024-02-10T07:44:07.513177+0000","last_active":"2024-02-10T07:44:36.343267+0000","last_peered":"2024-02-10T07:44:36.343267+0000","last_clean":"2024-02-10T07:44:36.343267+0000","last_became_active":"2024-02-10T07:44:07.512270+0000","last_became_peered":"2024-02-10T07:44:07.512270+0000","last_unstale":"2024-02-10T07:44:36.343267+0000","last_undegraded":"2024-02-10T07:44:36.343267+0000","last_fullsized":"2024-02-10T07:44:36.343267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:00:23.656408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342999+0000","last_change":"2024-02-10T07:44:09.523182+0000","last_active":"2024-02-10T07:44:36.342999+0000","last_peered":"2024-02-10T07:44:36.342999+0000","last_clean":"2024-02-10T07:44:36.342999+0000","last_became_active":"2024-02-10T07:44:09.522189+0000","last_became_peered":"2024-02-10T07:44:09.522189+0000","last_unstale":"2024-02-10T07:44:36.342999+0000","last_undegraded":"2024-02-10T07:44:36.342999+0000","last_fullsized":"2024-02-10T07:44:36.342999+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:57:12.167059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340511+0000","last_change":"2024-02-10T07:44:13.554804+0000","last_active":"2024-02-10T07:44:36.340511+0000","last_peered":"2024-02-10T07:44:36.340511+0000","last_clean":"2024-02-10T07:44:36.340511+0000","last_became_active":"2024-02-10T07:44:13.554446+0000","last_became_peered":"2024-02-10T07:44:13.554446+0000","last_unstale":"2024-02-10T07:44:36.340511+0000","last_undegraded":"2024-02-10T07:44:36.340511+0000","last_fullsized":"2024-02-10T07:44:36.340511+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:01:26.831537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339390+0000","last_change":"2024-02-10T07:44:11.536996+0000","last_active":"2024-02-10T07:44:36.339390+0000","last_peered":"2024-02-10T07:44:36.339390+0000","last_clean":"2024-02-10T07:44:36.339390+0000","last_became_active":"2024-02-10T07:44:11.536126+0000","last_became_peered":"2024-02-10T07:44:11.536126+0000","last_unstale":"2024-02-10T07:44:36.339390+0000","last_undegraded":"2024-02-10T07:44:36.339390+0000","last_fullsized":"2024-02-10T07:44:36.339390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:19:57.891667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342763+0000","last_change":"2024-02-10T07:44:07.512872+0000","last_active":"2024-02-10T07:44:36.342763+0000","last_peered":"2024-02-10T07:44:36.342763+0000","last_clean":"2024-02-10T07:44:36.342763+0000","last_became_active":"2024-02-10T07:44:07.512560+0000","last_became_peered":"2024-02-10T07:44:07.512560+0000","last_unstale":"2024-02-10T07:44:36.342763+0000","last_undegraded":"2024-02-10T07:44:36.342763+0000","last_fullsized":"2024-02-10T07:44:36.342763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:35:39.277816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343082+0000","last_change":"2024-02-10T07:44:09.522255+0000","last_active":"2024-02-10T07:44:36.343082+0000","last_peered":"2024-02-10T07:44:36.343082+0000","last_clean":"2024-02-10T07:44:36.343082+0000","last_became_active":"2024-02-10T07:44:09.521848+0000","last_became_peered":"2024-02-10T07:44:09.521848+0000","last_unstale":"2024-02-10T07:44:36.343082+0000","last_undegraded":"2024-02-10T07:44:36.343082+0000","last_fullsized":"2024-02-10T07:44:36.343082+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:35:07.033421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340974+0000","last_change":"2024-02-10T07:44:13.555337+0000","last_active":"2024-02-10T07:44:36.340974+0000","last_peered":"2024-02-10T07:44:36.340974+0000","last_clean":"2024-02-10T07:44:36.340974+0000","last_became_active":"2024-02-10T07:44:13.553637+0000","last_became_peered":"2024-02-10T07:44:13.553637+0000","last_unstale":"2024-02-10T07:44:36.340974+0000","last_undegraded":"2024-02-10T07:44:36.340974+0000","last_fullsized":"2024-02-10T07:44:36.340974+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:09:12.367800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562249+0000","last_change":"2024-02-10T07:44:11.538707+0000","last_active":"2024-02-10T07:44:37.562249+0000","last_peered":"2024-02-10T07:44:37.562249+0000","last_clean":"2024-02-10T07:44:37.562249+0000","last_became_active":"2024-02-10T07:44:11.538414+0000","last_became_peered":"2024-02-10T07:44:11.538414+0000","last_unstale":"2024-02-10T07:44:37.562249+0000","last_undegraded":"2024-02-10T07:44:37.562249+0000","last_fullsized":"2024-02-10T07:44:37.562249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:54:13.722018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341304+0000","last_change":"2024-02-10T07:44:07.512731+0000","last_active":"2024-02-10T07:44:36.341304+0000","last_peered":"2024-02-10T07:44:36.341304+0000","last_clean":"2024-02-10T07:44:36.341304+0000","last_became_active":"2024-02-10T07:44:07.512338+0000","last_became_peered":"2024-02-10T07:44:07.512338+0000","last_unstale":"2024-02-10T07:44:36.341304+0000","last_undegraded":"2024-02-10T07:44:36.341304+0000","last_fullsized":"2024-02-10T07:44:36.341304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:11:50.141235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339744+0000","last_change":"2024-02-10T07:44:09.524972+0000","last_active":"2024-02-10T07:44:36.339744+0000","last_peered":"2024-02-10T07:44:36.339744+0000","last_clean":"2024-02-10T07:44:36.339744+0000","last_became_active":"2024-02-10T07:44:09.524683+0000","last_became_peered":"2024-02-10T07:44:09.524683+0000","last_unstale":"2024-02-10T07:44:36.339744+0000","last_undegraded":"2024-02-10T07:44:36.339744+0000","last_fullsized":"2024-02-10T07:44:36.339744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:39:02.389958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340935+0000","last_change":"2024-02-10T07:44:13.555129+0000","last_active":"2024-02-10T07:44:36.340935+0000","last_peered":"2024-02-10T07:44:36.340935+0000","last_clean":"2024-02-10T07:44:36.340935+0000","last_became_active":"2024-02-10T07:44:13.554648+0000","last_became_peered":"2024-02-10T07:44:13.554648+0000","last_unstale":"2024-02-10T07:44:36.340935+0000","last_undegraded":"2024-02-10T07:44:36.340935+0000","last_fullsized":"2024-02-10T07:44:36.340935+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:35:42.035642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562307+0000","last_change":"2024-02-10T07:44:11.538072+0000","last_active":"2024-02-10T07:44:37.562307+0000","last_peered":"2024-02-10T07:44:37.562307+0000","last_clean":"2024-02-10T07:44:37.562307+0000","last_became_active":"2024-02-10T07:44:11.537669+0000","last_became_peered":"2024-02-10T07:44:11.537669+0000","last_unstale":"2024-02-10T07:44:37.562307+0000","last_undegraded":"2024-02-10T07:44:37.562307+0000","last_fullsized":"2024-02-10T07:44:37.562307+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:42.820531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340747+0000","last_change":"2024-02-10T07:44:07.510570+0000","last_active":"2024-02-10T07:44:36.340747+0000","last_peered":"2024-02-10T07:44:36.340747+0000","last_clean":"2024-02-10T07:44:36.340747+0000","last_became_active":"2024-02-10T07:44:07.510245+0000","last_became_peered":"2024-02-10T07:44:07.510245+0000","last_unstale":"2024-02-10T07:44:36.340747+0000","last_undegraded":"2024-02-10T07:44:36.340747+0000","last_fullsized":"2024-02-10T07:44:36.340747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:53:25.229381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342759+0000","last_change":"2024-02-10T07:44:09.520674+0000","last_active":"2024-02-10T07:44:36.342759+0000","last_peered":"2024-02-10T07:44:36.342759+0000","last_clean":"2024-02-10T07:44:36.342759+0000","last_became_active":"2024-02-10T07:44:09.520360+0000","last_became_peered":"2024-02-10T07:44:09.520360+0000","last_unstale":"2024-02-10T07:44:36.342759+0000","last_undegraded":"2024-02-10T07:44:36.342759+0000","last_fullsized":"2024-02-10T07:44:36.342759+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:08:35.506767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342733+0000","last_change":"2024-02-10T07:44:13.556851+0000","last_active":"2024-02-10T07:44:36.342733+0000","last_peered":"2024-02-10T07:44:36.342733+0000","last_clean":"2024-02-10T07:44:36.342733+0000","last_became_active":"2024-02-10T07:44:13.556481+0000","last_became_peered":"2024-02-10T07:44:13.556481+0000","last_unstale":"2024-02-10T07:44:36.342733+0000","last_undegraded":"2024-02-10T07:44:36.342733+0000","last_fullsized":"2024-02-10T07:44:36.342733+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:33.112634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341445+0000","last_change":"2024-02-10T07:44:11.538136+0000","last_active":"2024-02-10T07:44:36.341445+0000","last_peered":"2024-02-10T07:44:36.341445+0000","last_clean":"2024-02-10T07:44:36.341445+0000","last_became_active":"2024-02-10T07:44:11.537683+0000","last_became_peered":"2024-02-10T07:44:11.537683+0000","last_unstale":"2024-02-10T07:44:36.341445+0000","last_undegraded":"2024-02-10T07:44:36.341445+0000","last_fullsized":"2024-02-10T07:44:36.341445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:44:20.746410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340696+0000","last_change":"2024-02-10T07:44:07.514186+0000","last_active":"2024-02-10T07:44:36.340696+0000","last_peered":"2024-02-10T07:44:36.340696+0000","last_clean":"2024-02-10T07:44:36.340696+0000","last_became_active":"2024-02-10T07:44:07.513907+0000","last_became_peered":"2024-02-10T07:44:07.513907+0000","last_unstale":"2024-02-10T07:44:36.340696+0000","last_undegraded":"2024-02-10T07:44:36.340696+0000","last_fullsized":"2024-02-10T07:44:36.340696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:03:33.682218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340064+0000","last_change":"2024-02-10T07:44:09.524592+0000","last_active":"2024-02-10T07:44:36.340064+0000","last_peered":"2024-02-10T07:44:36.340064+0000","last_clean":"2024-02-10T07:44:36.340064+0000","last_became_active":"2024-02-10T07:44:09.524351+0000","last_became_peered":"2024-02-10T07:44:09.524351+0000","last_unstale":"2024-02-10T07:44:36.340064+0000","last_undegraded":"2024-02-10T07:44:36.340064+0000","last_fullsized":"2024-02-10T07:44:36.340064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:36:57.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341189+0000","last_change":"2024-02-10T07:44:13.560537+0000","last_active":"2024-02-10T07:44:36.341189+0000","last_peered":"2024-02-10T07:44:36.341189+0000","last_clean":"2024-02-10T07:44:36.341189+0000","last_became_active":"2024-02-10T07:44:13.560134+0000","last_became_peered":"2024-02-10T07:44:13.560134+0000","last_unstale":"2024-02-10T07:44:36.341189+0000","last_undegraded":"2024-02-10T07:44:36.341189+0000","last_fullsized":"2024-02-10T07:44:36.341189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:22:57.271941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341371+0000","last_change":"2024-02-10T07:44:11.535946+0000","last_active":"2024-02-10T07:44:36.341371+0000","last_peered":"2024-02-10T07:44:36.341371+0000","last_clean":"2024-02-10T07:44:36.341371+0000","last_became_active":"2024-02-10T07:44:11.535603+0000","last_became_peered":"2024-02-10T07:44:11.535603+0000","last_unstale":"2024-02-10T07:44:36.341371+0000","last_undegraded":"2024-02-10T07:44:36.341371+0000","last_fullsized":"2024-02-10T07:44:36.341371+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:17:49.836513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340609+0000","last_change":"2024-02-10T07:44:07.513566+0000","last_active":"2024-02-10T07:44:36.340609+0000","last_peered":"2024-02-10T07:44:36.340609+0000","last_clean":"2024-02-10T07:44:36.340609+0000","last_became_active":"2024-02-10T07:44:07.513283+0000","last_became_peered":"2024-02-10T07:44:07.513283+0000","last_unstale":"2024-02-10T07:44:36.340609+0000","last_undegraded":"2024-02-10T07:44:36.340609+0000","last_fullsized":"2024-02-10T07:44:36.340609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:08:04.575929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340502+0000","last_change":"2024-02-10T07:44:09.521875+0000","last_active":"2024-02-10T07:44:36.340502+0000","last_peered":"2024-02-10T07:44:36.340502+0000","last_clean":"2024-02-10T07:44:36.340502+0000","last_became_active":"2024-02-10T07:44:09.521527+0000","last_became_peered":"2024-02-10T07:44:09.521527+0000","last_unstale":"2024-02-10T07:44:36.340502+0000","last_undegraded":"2024-02-10T07:44:36.340502+0000","last_fullsized":"2024-02-10T07:44:36.340502+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:22:02.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342120+0000","last_change":"2024-02-10T07:44:13.556643+0000","last_active":"2024-02-10T07:44:36.342120+0000","last_peered":"2024-02-10T07:44:36.342120+0000","last_clean":"2024-02-10T07:44:36.342120+0000","last_became_active":"2024-02-10T07:44:13.556356+0000","last_became_peered":"2024-02-10T07:44:13.556356+0000","last_unstale":"2024-02-10T07:44:36.342120+0000","last_undegraded":"2024-02-10T07:44:36.342120+0000","last_fullsized":"2024-02-10T07:44:36.342120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:31:01.042207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343091+0000","last_change":"2024-02-10T07:44:07.510878+0000","last_active":"2024-02-10T07:44:36.343091+0000","last_peered":"2024-02-10T07:44:36.343091+0000","last_clean":"2024-02-10T07:44:36.343091+0000","last_became_active":"2024-02-10T07:44:07.510596+0000","last_became_peered":"2024-02-10T07:44:07.510596+0000","last_unstale":"2024-02-10T07:44:36.343091+0000","last_undegraded":"2024-02-10T07:44:36.343091+0000","last_fullsized":"2024-02-10T07:44:36.343091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:03:56.341218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342487+0000","last_change":"2024-02-10T07:44:09.525951+0000","last_active":"2024-02-10T07:44:36.342487+0000","last_peered":"2024-02-10T07:44:36.342487+0000","last_clean":"2024-02-10T07:44:36.342487+0000","last_became_active":"2024-02-10T07:44:09.525725+0000","last_became_peered":"2024-02-10T07:44:09.525725+0000","last_unstale":"2024-02-10T07:44:36.342487+0000","last_undegraded":"2024-02-10T07:44:36.342487+0000","last_fullsized":"2024-02-10T07:44:36.342487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:13.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562358+0000","last_change":"2024-02-10T07:44:11.538707+0000","last_active":"2024-02-10T07:44:37.562358+0000","last_peered":"2024-02-10T07:44:37.562358+0000","last_clean":"2024-02-10T07:44:37.562358+0000","last_became_active":"2024-02-10T07:44:11.538433+0000","last_became_peered":"2024-02-10T07:44:11.538433+0000","last_unstale":"2024-02-10T07:44:37.562358+0000","last_undegraded":"2024-02-10T07:44:37.562358+0000","last_fullsized":"2024-02-10T07:44:37.562358+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:47:49.712669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339299+0000","last_change":"2024-02-10T07:44:13.555289+0000","last_active":"2024-02-10T07:44:36.339299+0000","last_peered":"2024-02-10T07:44:36.339299+0000","last_clean":"2024-02-10T07:44:36.339299+0000","last_became_active":"2024-02-10T07:44:13.554789+0000","last_became_peered":"2024-02-10T07:44:13.554789+0000","last_unstale":"2024-02-10T07:44:36.339299+0000","last_undegraded":"2024-02-10T07:44:36.339299+0000","last_fullsized":"2024-02-10T07:44:36.339299+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:11.835869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341244+0000","last_change":"2024-02-10T07:44:07.511613+0000","last_active":"2024-02-10T07:44:36.341244+0000","last_peered":"2024-02-10T07:44:36.341244+0000","last_clean":"2024-02-10T07:44:36.341244+0000","last_became_active":"2024-02-10T07:44:07.510220+0000","last_became_peered":"2024-02-10T07:44:07.510220+0000","last_unstale":"2024-02-10T07:44:36.341244+0000","last_undegraded":"2024-02-10T07:44:36.341244+0000","last_fullsized":"2024-02-10T07:44:36.341244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:34:50.227635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":53,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562253+0000","last_change":"2024-02-10T07:44:09.520915+0000","last_active":"2024-02-10T07:44:37.562253+0000","last_peered":"2024-02-10T07:44:37.562253+0000","last_clean":"2024-02-10T07:44:37.562253+0000","last_became_active":"2024-02-10T07:44:09.520587+0000","last_became_peered":"2024-02-10T07:44:09.520587+0000","last_unstale":"2024-02-10T07:44:37.562253+0000","last_undegraded":"2024-02-10T07:44:37.562253+0000","last_fullsized":"2024-02-10T07:44:37.562253+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:50:49.331970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340508+0000","last_change":"2024-02-10T07:44:11.537804+0000","last_active":"2024-02-10T07:44:36.340508+0000","last_peered":"2024-02-10T07:44:36.340508+0000","last_clean":"2024-02-10T07:44:36.340508+0000","last_became_active":"2024-02-10T07:44:11.537443+0000","last_became_peered":"2024-02-10T07:44:11.537443+0000","last_unstale":"2024-02-10T07:44:36.340508+0000","last_undegraded":"2024-02-10T07:44:36.340508+0000","last_fullsized":"2024-02-10T07:44:36.340508+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:21:25.733201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340398+0000","last_change":"2024-02-10T07:44:13.553313+0000","last_active":"2024-02-10T07:44:36.340398+0000","last_peered":"2024-02-10T07:44:36.340398+0000","last_clean":"2024-02-10T07:44:36.340398+0000","last_became_active":"2024-02-10T07:44:13.552865+0000","last_became_peered":"2024-02-10T07:44:13.552865+0000","last_unstale":"2024-02-10T07:44:36.340398+0000","last_undegraded":"2024-02-10T07:44:36.340398+0000","last_fullsized":"2024-02-10T07:44:36.340398+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:29:54.822325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341017+0000","last_change":"2024-02-10T07:44:07.513499+0000","last_active":"2024-02-10T07:44:36.341017+0000","last_peered":"2024-02-10T07:44:36.341017+0000","last_clean":"2024-02-10T07:44:36.341017+0000","last_became_active":"2024-02-10T07:44:07.513238+0000","last_became_peered":"2024-02-10T07:44:07.513238+0000","last_unstale":"2024-02-10T07:44:36.341017+0000","last_undegraded":"2024-02-10T07:44:36.341017+0000","last_fullsized":"2024-02-10T07:44:36.341017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:34:09.731229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342904+0000","last_change":"2024-02-10T07:44:09.520474+0000","last_active":"2024-02-10T07:44:36.342904+0000","last_peered":"2024-02-10T07:44:36.342904+0000","last_clean":"2024-02-10T07:44:36.342904+0000","last_became_active":"2024-02-10T07:44:09.520077+0000","last_became_peered":"2024-02-10T07:44:09.520077+0000","last_unstale":"2024-02-10T07:44:36.342904+0000","last_undegraded":"2024-02-10T07:44:36.342904+0000","last_fullsized":"2024-02-10T07:44:36.342904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:38:39.985450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341063+0000","last_change":"2024-02-10T07:44:11.537700+0000","last_active":"2024-02-10T07:44:36.341063+0000","last_peered":"2024-02-10T07:44:36.341063+0000","last_clean":"2024-02-10T07:44:36.341063+0000","last_became_active":"2024-02-10T07:44:11.537339+0000","last_became_peered":"2024-02-10T07:44:11.537339+0000","last_unstale":"2024-02-10T07:44:36.341063+0000","last_undegraded":"2024-02-10T07:44:36.341063+0000","last_fullsized":"2024-02-10T07:44:36.341063+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:11:59.196770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342257+0000","last_change":"2024-02-10T07:44:13.560152+0000","last_active":"2024-02-10T07:44:36.342257+0000","last_peered":"2024-02-10T07:44:36.342257+0000","last_clean":"2024-02-10T07:44:36.342257+0000","last_became_active":"2024-02-10T07:44:13.559756+0000","last_became_peered":"2024-02-10T07:44:13.559756+0000","last_unstale":"2024-02-10T07:44:36.342257+0000","last_undegraded":"2024-02-10T07:44:36.342257+0000","last_fullsized":"2024-02-10T07:44:36.342257+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:22:30.661298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340087+0000","last_change":"2024-02-10T07:44:09.523085+0000","last_active":"2024-02-10T07:44:36.340087+0000","last_peered":"2024-02-10T07:44:36.340087+0000","last_clean":"2024-02-10T07:44:36.340087+0000","last_became_active":"2024-02-10T07:44:09.522442+0000","last_became_peered":"2024-02-10T07:44:09.522442+0000","last_unstale":"2024-02-10T07:44:36.340087+0000","last_undegraded":"2024-02-10T07:44:36.340087+0000","last_fullsized":"2024-02-10T07:44:36.340087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:18:55.612922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339304+0000","last_change":"2024-02-10T07:44:07.512395+0000","last_active":"2024-02-10T07:44:36.339304+0000","last_peered":"2024-02-10T07:44:36.339304+0000","last_clean":"2024-02-10T07:44:36.339304+0000","last_became_active":"2024-02-10T07:44:07.511570+0000","last_became_peered":"2024-02-10T07:44:07.511570+0000","last_unstale":"2024-02-10T07:44:36.339304+0000","last_undegraded":"2024-02-10T07:44:36.339304+0000","last_fullsized":"2024-02-10T07:44:36.339304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:32:05.713726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341733+0000","last_change":"2024-02-10T07:44:11.539349+0000","last_active":"2024-02-10T07:44:36.341733+0000","last_peered":"2024-02-10T07:44:36.341733+0000","last_clean":"2024-02-10T07:44:36.341733+0000","last_became_active":"2024-02-10T07:44:11.537888+0000","last_became_peered":"2024-02-10T07:44:11.537888+0000","last_unstale":"2024-02-10T07:44:36.341733+0000","last_undegraded":"2024-02-10T07:44:36.341733+0000","last_fullsized":"2024-02-10T07:44:36.341733+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:12:57.192641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341100+0000","last_change":"2024-02-10T07:44:13.555785+0000","last_active":"2024-02-10T07:44:36.341100+0000","last_peered":"2024-02-10T07:44:36.341100+0000","last_clean":"2024-02-10T07:44:36.341100+0000","last_became_active":"2024-02-10T07:44:13.555176+0000","last_became_peered":"2024-02-10T07:44:13.555176+0000","last_unstale":"2024-02-10T07:44:36.341100+0000","last_undegraded":"2024-02-10T07:44:36.341100+0000","last_fullsized":"2024-02-10T07:44:36.341100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:12:59.832906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339550+0000","last_change":"2024-02-10T07:44:09.527329+0000","last_active":"2024-02-10T07:44:36.339550+0000","last_peered":"2024-02-10T07:44:36.339550+0000","last_clean":"2024-02-10T07:44:36.339550+0000","last_became_active":"2024-02-10T07:44:09.526246+0000","last_became_peered":"2024-02-10T07:44:09.526246+0000","last_unstale":"2024-02-10T07:44:36.339550+0000","last_undegraded":"2024-02-10T07:44:36.339550+0000","last_fullsized":"2024-02-10T07:44:36.339550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:00:22.023550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341936+0000","last_change":"2024-02-10T07:44:07.512537+0000","last_active":"2024-02-10T07:44:36.341936+0000","last_peered":"2024-02-10T07:44:36.341936+0000","last_clean":"2024-02-10T07:44:36.341936+0000","last_became_active":"2024-02-10T07:44:07.512249+0000","last_became_peered":"2024-02-10T07:44:07.512249+0000","last_unstale":"2024-02-10T07:44:36.341936+0000","last_undegraded":"2024-02-10T07:44:36.341936+0000","last_fullsized":"2024-02-10T07:44:36.341936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:13.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:38.438336+0000","last_change":"2024-02-10T07:43:55.035647+0000","last_active":"2024-02-10T07:44:38.438336+0000","last_peered":"2024-02-10T07:44:38.438336+0000","last_clean":"2024-02-10T07:44:38.438336+0000","last_became_active":"2024-02-10T07:43:55.035360+0000","last_became_peered":"2024-02-10T07:43:55.035360+0000","last_unstale":"2024-02-10T07:44:38.438336+0000","last_undegraded":"2024-02-10T07:44:38.438336+0000","last_fullsized":"2024-02-10T07:44:38.438336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:43:54.028946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:43:54.028946+0000","last_clean_scrub_stamp":"2024-02-10T07:43:54.028946+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:18.898006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341385+0000","last_change":"2024-02-10T07:44:11.543205+0000","last_active":"2024-02-10T07:44:36.341385+0000","last_peered":"2024-02-10T07:44:36.341385+0000","last_clean":"2024-02-10T07:44:36.341385+0000","last_became_active":"2024-02-10T07:44:11.542603+0000","last_became_peered":"2024-02-10T07:44:11.542603+0000","last_unstale":"2024-02-10T07:44:36.341385+0000","last_undegraded":"2024-02-10T07:44:36.341385+0000","last_fullsized":"2024-02-10T07:44:36.341385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:24:56.688038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341668+0000","last_change":"2024-02-10T07:44:13.555726+0000","last_active":"2024-02-10T07:44:36.341668+0000","last_peered":"2024-02-10T07:44:36.341668+0000","last_clean":"2024-02-10T07:44:36.341668+0000","last_became_active":"2024-02-10T07:44:13.554674+0000","last_became_peered":"2024-02-10T07:44:13.554674+0000","last_unstale":"2024-02-10T07:44:36.341668+0000","last_undegraded":"2024-02-10T07:44:36.341668+0000","last_fullsized":"2024-02-10T07:44:36.341668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:40:34.479843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339559+0000","last_change":"2024-02-10T07:44:09.526625+0000","last_active":"2024-02-10T07:44:36.339559+0000","last_peered":"2024-02-10T07:44:36.339559+0000","last_clean":"2024-02-10T07:44:36.339559+0000","last_became_active":"2024-02-10T07:44:09.526250+0000","last_became_peered":"2024-02-10T07:44:09.526250+0000","last_unstale":"2024-02-10T07:44:36.339559+0000","last_undegraded":"2024-02-10T07:44:36.339559+0000","last_fullsized":"2024-02-10T07:44:36.339559+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:29:24.829423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340075+0000","last_change":"2024-02-10T07:44:07.512818+0000","last_active":"2024-02-10T07:44:36.340075+0000","last_peered":"2024-02-10T07:44:36.340075+0000","last_clean":"2024-02-10T07:44:36.340075+0000","last_became_active":"2024-02-10T07:44:07.512486+0000","last_became_peered":"2024-02-10T07:44:07.512486+0000","last_unstale":"2024-02-10T07:44:36.340075+0000","last_undegraded":"2024-02-10T07:44:36.340075+0000","last_fullsized":"2024-02-10T07:44:36.340075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.343865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341418+0000","last_change":"2024-02-10T07:44:11.541376+0000","last_active":"2024-02-10T07:44:36.341418+0000","last_peered":"2024-02-10T07:44:36.341418+0000","last_clean":"2024-02-10T07:44:36.341418+0000","last_became_active":"2024-02-10T07:44:11.541061+0000","last_became_peered":"2024-02-10T07:44:11.541061+0000","last_unstale":"2024-02-10T07:44:36.341418+0000","last_undegraded":"2024-02-10T07:44:36.341418+0000","last_fullsized":"2024-02-10T07:44:36.341418+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:46:04.124370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340748+0000","last_change":"2024-02-10T07:44:13.554869+0000","last_active":"2024-02-10T07:44:36.340748+0000","last_peered":"2024-02-10T07:44:36.340748+0000","last_clean":"2024-02-10T07:44:36.340748+0000","last_became_active":"2024-02-10T07:44:13.554481+0000","last_became_peered":"2024-02-10T07:44:13.554481+0000","last_unstale":"2024-02-10T07:44:36.340748+0000","last_undegraded":"2024-02-10T07:44:36.340748+0000","last_fullsized":"2024-02-10T07:44:36.340748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:25:33.264675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343222+0000","last_change":"2024-02-10T07:44:07.513599+0000","last_active":"2024-02-10T07:44:36.343222+0000","last_peered":"2024-02-10T07:44:36.343222+0000","last_clean":"2024-02-10T07:44:36.343222+0000","last_became_active":"2024-02-10T07:44:07.512707+0000","last_became_peered":"2024-02-10T07:44:07.512707+0000","last_unstale":"2024-02-10T07:44:36.343222+0000","last_undegraded":"2024-02-10T07:44:36.343222+0000","last_fullsized":"2024-02-10T07:44:36.343222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:39:26.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":79,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562186+0000","last_change":"2024-02-10T07:44:09.526443+0000","last_active":"2024-02-10T07:44:37.562186+0000","last_peered":"2024-02-10T07:44:37.562186+0000","last_clean":"2024-02-10T07:44:37.562186+0000","last_became_active":"2024-02-10T07:44:09.526059+0000","last_became_peered":"2024-02-10T07:44:09.526059+0000","last_unstale":"2024-02-10T07:44:37.562186+0000","last_undegraded":"2024-02-10T07:44:37.562186+0000","last_fullsized":"2024-02-10T07:44:37.562186+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:09:15.908095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562335+0000","last_change":"2024-02-10T07:44:11.537809+0000","last_active":"2024-02-10T07:44:37.562335+0000","last_peered":"2024-02-10T07:44:37.562335+0000","last_clean":"2024-02-10T07:44:37.562335+0000","last_became_active":"2024-02-10T07:44:11.537496+0000","last_became_peered":"2024-02-10T07:44:11.537496+0000","last_unstale":"2024-02-10T07:44:37.562335+0000","last_undegraded":"2024-02-10T07:44:37.562335+0000","last_fullsized":"2024-02-10T07:44:37.562335+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:59:02.165808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340322+0000","last_change":"2024-02-10T07:44:13.554740+0000","last_active":"2024-02-10T07:44:36.340322+0000","last_peered":"2024-02-10T07:44:36.340322+0000","last_clean":"2024-02-10T07:44:36.340322+0000","last_became_active":"2024-02-10T07:44:13.554094+0000","last_became_peered":"2024-02-10T07:44:13.554094+0000","last_unstale":"2024-02-10T07:44:36.340322+0000","last_undegraded":"2024-02-10T07:44:36.340322+0000","last_fullsized":"2024-02-10T07:44:36.340322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:43:38.357448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342158+0000","last_change":"2024-02-10T07:44:07.511566+0000","last_active":"2024-02-10T07:44:36.342158+0000","last_peered":"2024-02-10T07:44:36.342158+0000","last_clean":"2024-02-10T07:44:36.342158+0000","last_became_active":"2024-02-10T07:44:07.510638+0000","last_became_peered":"2024-02-10T07:44:07.510638+0000","last_unstale":"2024-02-10T07:44:36.342158+0000","last_undegraded":"2024-02-10T07:44:36.342158+0000","last_fullsized":"2024-02-10T07:44:36.342158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:47:13.935384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342211+0000","last_change":"2024-02-10T07:44:09.527972+0000","last_active":"2024-02-10T07:44:36.342211+0000","last_peered":"2024-02-10T07:44:36.342211+0000","last_clean":"2024-02-10T07:44:36.342211+0000","last_became_active":"2024-02-10T07:44:09.527590+0000","last_became_peered":"2024-02-10T07:44:09.527590+0000","last_unstale":"2024-02-10T07:44:36.342211+0000","last_undegraded":"2024-02-10T07:44:36.342211+0000","last_fullsized":"2024-02-10T07:44:36.342211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:40:43.036140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562175+0000","last_change":"2024-02-10T07:44:11.538135+0000","last_active":"2024-02-10T07:44:37.562175+0000","last_peered":"2024-02-10T07:44:37.562175+0000","last_clean":"2024-02-10T07:44:37.562175+0000","last_became_active":"2024-02-10T07:44:11.537399+0000","last_became_peered":"2024-02-10T07:44:11.537399+0000","last_unstale":"2024-02-10T07:44:37.562175+0000","last_undegraded":"2024-02-10T07:44:37.562175+0000","last_fullsized":"2024-02-10T07:44:37.562175+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:44:50.343670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342069+0000","last_change":"2024-02-10T07:44:13.555904+0000","last_active":"2024-02-10T07:44:36.342069+0000","last_peered":"2024-02-10T07:44:36.342069+0000","last_clean":"2024-02-10T07:44:36.342069+0000","last_became_active":"2024-02-10T07:44:13.555614+0000","last_became_peered":"2024-02-10T07:44:13.555614+0000","last_unstale":"2024-02-10T07:44:36.342069+0000","last_undegraded":"2024-02-10T07:44:36.342069+0000","last_fullsized":"2024-02-10T07:44:36.342069+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:48:54.884304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342099+0000","last_change":"2024-02-10T07:44:07.514113+0000","last_active":"2024-02-10T07:44:36.342099+0000","last_peered":"2024-02-10T07:44:36.342099+0000","last_clean":"2024-02-10T07:44:36.342099+0000","last_became_active":"2024-02-10T07:44:07.513832+0000","last_became_peered":"2024-02-10T07:44:07.513832+0000","last_unstale":"2024-02-10T07:44:36.342099+0000","last_undegraded":"2024-02-10T07:44:36.342099+0000","last_fullsized":"2024-02-10T07:44:36.342099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:43:27.602809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341561+0000","last_change":"2024-02-10T07:44:09.527158+0000","last_active":"2024-02-10T07:44:36.341561+0000","last_peered":"2024-02-10T07:44:36.341561+0000","last_clean":"2024-02-10T07:44:36.341561+0000","last_became_active":"2024-02-10T07:44:09.526342+0000","last_became_peered":"2024-02-10T07:44:09.526342+0000","last_unstale":"2024-02-10T07:44:36.341561+0000","last_undegraded":"2024-02-10T07:44:36.341561+0000","last_fullsized":"2024-02-10T07:44:36.341561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:38:25.088256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340281+0000","last_change":"2024-02-10T07:44:11.538858+0000","last_active":"2024-02-10T07:44:36.340281+0000","last_peered":"2024-02-10T07:44:36.340281+0000","last_clean":"2024-02-10T07:44:36.340281+0000","last_became_active":"2024-02-10T07:44:11.538620+0000","last_became_peered":"2024-02-10T07:44:11.538620+0000","last_unstale":"2024-02-10T07:44:36.340281+0000","last_undegraded":"2024-02-10T07:44:36.340281+0000","last_fullsized":"2024-02-10T07:44:36.340281+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:24:26.479799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341504+0000","last_change":"2024-02-10T07:44:13.555720+0000","last_active":"2024-02-10T07:44:36.341504+0000","last_peered":"2024-02-10T07:44:36.341504+0000","last_clean":"2024-02-10T07:44:36.341504+0000","last_became_active":"2024-02-10T07:44:13.554995+0000","last_became_peered":"2024-02-10T07:44:13.554995+0000","last_unstale":"2024-02-10T07:44:36.341504+0000","last_undegraded":"2024-02-10T07:44:36.341504+0000","last_fullsized":"2024-02-10T07:44:36.341504+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:52:41.761034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":35,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562212+0000","last_change":"2024-02-10T07:44:11.542900+0000","last_active":"2024-02-10T07:44:37.562212+0000","last_peered":"2024-02-10T07:44:37.562212+0000","last_clean":"2024-02-10T07:44:37.562212+0000","last_became_active":"2024-02-10T07:44:11.541988+0000","last_became_peered":"2024-02-10T07:44:11.541988+0000","last_unstale":"2024-02-10T07:44:37.562212+0000","last_undegraded":"2024-02-10T07:44:37.562212+0000","last_fullsized":"2024-02-10T07:44:37.562212+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:20:46.650085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340300+0000","last_change":"2024-02-10T07:44:07.514719+0000","last_active":"2024-02-10T07:44:36.340300+0000","last_peered":"2024-02-10T07:44:36.340300+0000","last_clean":"2024-02-10T07:44:36.340300+0000","last_became_active":"2024-02-10T07:44:07.513918+0000","last_became_peered":"2024-02-10T07:44:07.513918+0000","last_unstale":"2024-02-10T07:44:36.340300+0000","last_undegraded":"2024-02-10T07:44:36.340300+0000","last_fullsized":"2024-02-10T07:44:36.340300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:44:30.306094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340633+0000","last_change":"2024-02-10T07:44:09.522028+0000","last_active":"2024-02-10T07:44:36.340633+0000","last_peered":"2024-02-10T07:44:36.340633+0000","last_clean":"2024-02-10T07:44:36.340633+0000","last_became_active":"2024-02-10T07:44:09.521610+0000","last_became_peered":"2024-02-10T07:44:09.521610+0000","last_unstale":"2024-02-10T07:44:36.340633+0000","last_undegraded":"2024-02-10T07:44:36.340633+0000","last_fullsized":"2024-02-10T07:44:36.340633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:18.457156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341565+0000","last_change":"2024-02-10T07:44:13.556306+0000","last_active":"2024-02-10T07:44:36.341565+0000","last_peered":"2024-02-10T07:44:36.341565+0000","last_clean":"2024-02-10T07:44:36.341565+0000","last_became_active":"2024-02-10T07:44:13.555458+0000","last_became_peered":"2024-02-10T07:44:13.555458+0000","last_unstale":"2024-02-10T07:44:36.341565+0000","last_undegraded":"2024-02-10T07:44:36.341565+0000","last_fullsized":"2024-02-10T07:44:36.341565+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:17:19.881762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341542+0000","last_change":"2024-02-10T07:44:11.537016+0000","last_active":"2024-02-10T07:44:36.341542+0000","last_peered":"2024-02-10T07:44:36.341542+0000","last_clean":"2024-02-10T07:44:36.341542+0000","last_became_active":"2024-02-10T07:44:11.536487+0000","last_became_peered":"2024-02-10T07:44:11.536487+0000","last_unstale":"2024-02-10T07:44:36.341542+0000","last_undegraded":"2024-02-10T07:44:36.341542+0000","last_fullsized":"2024-02-10T07:44:36.341542+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:02:55.450979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339427+0000","last_change":"2024-02-10T07:44:07.509499+0000","last_active":"2024-02-10T07:44:36.339427+0000","last_peered":"2024-02-10T07:44:36.339427+0000","last_clean":"2024-02-10T07:44:36.339427+0000","last_became_active":"2024-02-10T07:44:07.509038+0000","last_became_peered":"2024-02-10T07:44:07.509038+0000","last_unstale":"2024-02-10T07:44:36.339427+0000","last_undegraded":"2024-02-10T07:44:36.339427+0000","last_fullsized":"2024-02-10T07:44:36.339427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:55:24.697151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340703+0000","last_change":"2024-02-10T07:44:09.522567+0000","last_active":"2024-02-10T07:44:36.340703+0000","last_peered":"2024-02-10T07:44:36.340703+0000","last_clean":"2024-02-10T07:44:36.340703+0000","last_became_active":"2024-02-10T07:44:09.522239+0000","last_became_peered":"2024-02-10T07:44:09.522239+0000","last_unstale":"2024-02-10T07:44:36.340703+0000","last_undegraded":"2024-02-10T07:44:36.340703+0000","last_fullsized":"2024-02-10T07:44:36.340703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:35:29.808886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342861+0000","last_change":"2024-02-10T07:44:13.555899+0000","last_active":"2024-02-10T07:44:36.342861+0000","last_peered":"2024-02-10T07:44:36.342861+0000","last_clean":"2024-02-10T07:44:36.342861+0000","last_became_active":"2024-02-10T07:44:13.555267+0000","last_became_peered":"2024-02-10T07:44:13.555267+0000","last_unstale":"2024-02-10T07:44:36.342861+0000","last_undegraded":"2024-02-10T07:44:36.342861+0000","last_fullsized":"2024-02-10T07:44:36.342861+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:05:55.378474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341105+0000","last_change":"2024-02-10T07:44:07.509885+0000","last_active":"2024-02-10T07:44:36.341105+0000","last_peered":"2024-02-10T07:44:36.341105+0000","last_clean":"2024-02-10T07:44:36.341105+0000","last_became_active":"2024-02-10T07:44:07.509616+0000","last_became_peered":"2024-02-10T07:44:07.509616+0000","last_unstale":"2024-02-10T07:44:36.341105+0000","last_undegraded":"2024-02-10T07:44:36.341105+0000","last_fullsized":"2024-02-10T07:44:36.341105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:27:36.467289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339931+0000","last_change":"2024-02-10T07:44:09.526656+0000","last_active":"2024-02-10T07:44:36.339931+0000","last_peered":"2024-02-10T07:44:36.339931+0000","last_clean":"2024-02-10T07:44:36.339931+0000","last_became_active":"2024-02-10T07:44:09.526331+0000","last_became_peered":"2024-02-10T07:44:09.526331+0000","last_unstale":"2024-02-10T07:44:36.339931+0000","last_undegraded":"2024-02-10T07:44:36.339931+0000","last_fullsized":"2024-02-10T07:44:36.339931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:12:44.951278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341972+0000","last_change":"2024-02-10T07:44:11.542972+0000","last_active":"2024-02-10T07:44:36.341972+0000","last_peered":"2024-02-10T07:44:36.341972+0000","last_clean":"2024-02-10T07:44:36.341972+0000","last_became_active":"2024-02-10T07:44:11.542621+0000","last_became_peered":"2024-02-10T07:44:11.542621+0000","last_unstale":"2024-02-10T07:44:36.341972+0000","last_undegraded":"2024-02-10T07:44:36.341972+0000","last_fullsized":"2024-02-10T07:44:36.341972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:39:02.475530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340672+0000","last_change":"2024-02-10T07:44:13.555488+0000","last_active":"2024-02-10T07:44:36.340672+0000","last_peered":"2024-02-10T07:44:36.340672+0000","last_clean":"2024-02-10T07:44:36.340672+0000","last_became_active":"2024-02-10T07:44:13.555148+0000","last_became_peered":"2024-02-10T07:44:13.555148+0000","last_unstale":"2024-02-10T07:44:36.340672+0000","last_undegraded":"2024-02-10T07:44:36.340672+0000","last_fullsized":"2024-02-10T07:44:36.340672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:10.111608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:37.562168+0000","last_change":"2024-02-10T07:44:11.543200+0000","last_active":"2024-02-10T07:44:37.562168+0000","last_peered":"2024-02-10T07:44:37.562168+0000","last_clean":"2024-02-10T07:44:37.562168+0000","last_became_active":"2024-02-10T07:44:11.541833+0000","last_became_peered":"2024-02-10T07:44:11.541833+0000","last_unstale":"2024-02-10T07:44:37.562168+0000","last_undegraded":"2024-02-10T07:44:37.562168+0000","last_fullsized":"2024-02-10T07:44:37.562168+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:41:12.386529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341841+0000","last_change":"2024-02-10T07:44:07.509973+0000","last_active":"2024-02-10T07:44:36.341841+0000","last_peered":"2024-02-10T07:44:36.341841+0000","last_clean":"2024-02-10T07:44:36.341841+0000","last_became_active":"2024-02-10T07:44:07.509704+0000","last_became_peered":"2024-02-10T07:44:07.509704+0000","last_unstale":"2024-02-10T07:44:36.341841+0000","last_undegraded":"2024-02-10T07:44:36.341841+0000","last_fullsized":"2024-02-10T07:44:36.341841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:38:30.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341176+0000","last_change":"2024-02-10T07:44:09.521999+0000","last_active":"2024-02-10T07:44:36.341176+0000","last_peered":"2024-02-10T07:44:36.341176+0000","last_clean":"2024-02-10T07:44:36.341176+0000","last_became_active":"2024-02-10T07:44:09.521622+0000","last_became_peered":"2024-02-10T07:44:09.521622+0000","last_unstale":"2024-02-10T07:44:36.341176+0000","last_undegraded":"2024-02-10T07:44:36.341176+0000","last_fullsized":"2024-02-10T07:44:36.341176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:38.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341637+0000","last_change":"2024-02-10T07:44:13.555408+0000","last_active":"2024-02-10T07:44:36.341637+0000","last_peered":"2024-02-10T07:44:36.341637+0000","last_clean":"2024-02-10T07:44:36.341637+0000","last_became_active":"2024-02-10T07:44:13.555112+0000","last_became_peered":"2024-02-10T07:44:13.555112+0000","last_unstale":"2024-02-10T07:44:36.341637+0000","last_undegraded":"2024-02-10T07:44:36.341637+0000","last_fullsized":"2024-02-10T07:44:36.341637+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:13:16.941128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342492+0000","last_change":"2024-02-10T07:44:07.510831+0000","last_active":"2024-02-10T07:44:36.342492+0000","last_peered":"2024-02-10T07:44:36.342492+0000","last_clean":"2024-02-10T07:44:36.342492+0000","last_became_active":"2024-02-10T07:44:07.510539+0000","last_became_peered":"2024-02-10T07:44:07.510539+0000","last_unstale":"2024-02-10T07:44:36.342492+0000","last_undegraded":"2024-02-10T07:44:36.342492+0000","last_fullsized":"2024-02-10T07:44:36.342492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:58:30.733385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341727+0000","last_change":"2024-02-10T07:44:09.522588+0000","last_active":"2024-02-10T07:44:36.341727+0000","last_peered":"2024-02-10T07:44:36.341727+0000","last_clean":"2024-02-10T07:44:36.341727+0000","last_became_active":"2024-02-10T07:44:09.522101+0000","last_became_peered":"2024-02-10T07:44:09.522101+0000","last_unstale":"2024-02-10T07:44:36.341727+0000","last_undegraded":"2024-02-10T07:44:36.341727+0000","last_fullsized":"2024-02-10T07:44:36.341727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:51:40.634580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343272+0000","last_change":"2024-02-10T07:44:11.543761+0000","last_active":"2024-02-10T07:44:36.343272+0000","last_peered":"2024-02-10T07:44:36.343272+0000","last_clean":"2024-02-10T07:44:36.343272+0000","last_became_active":"2024-02-10T07:44:11.542721+0000","last_became_peered":"2024-02-10T07:44:11.542721+0000","last_unstale":"2024-02-10T07:44:36.343272+0000","last_undegraded":"2024-02-10T07:44:36.343272+0000","last_fullsized":"2024-02-10T07:44:36.343272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:24:08.373513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338869+0000","last_change":"2024-02-10T07:44:13.555283+0000","last_active":"2024-02-10T07:44:36.338869+0000","last_peered":"2024-02-10T07:44:36.338869+0000","last_clean":"2024-02-10T07:44:36.338869+0000","last_became_active":"2024-02-10T07:44:13.554841+0000","last_became_peered":"2024-02-10T07:44:13.554841+0000","last_unstale":"2024-02-10T07:44:36.338869+0000","last_undegraded":"2024-02-10T07:44:36.338869+0000","last_fullsized":"2024-02-10T07:44:36.338869+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:07.897574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343035+0000","last_change":"2024-02-10T07:44:07.513531+0000","last_active":"2024-02-10T07:44:36.343035+0000","last_peered":"2024-02-10T07:44:36.343035+0000","last_clean":"2024-02-10T07:44:36.343035+0000","last_became_active":"2024-02-10T07:44:07.512810+0000","last_became_peered":"2024-02-10T07:44:07.512810+0000","last_unstale":"2024-02-10T07:44:36.343035+0000","last_undegraded":"2024-02-10T07:44:36.343035+0000","last_fullsized":"2024-02-10T07:44:36.343035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:25:29.034573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342864+0000","last_change":"2024-02-10T07:44:09.523164+0000","last_active":"2024-02-10T07:44:36.342864+0000","last_peered":"2024-02-10T07:44:36.342864+0000","last_clean":"2024-02-10T07:44:36.342864+0000","last_became_active":"2024-02-10T07:44:09.522703+0000","last_became_peered":"2024-02-10T07:44:09.522703+0000","last_unstale":"2024-02-10T07:44:36.342864+0000","last_undegraded":"2024-02-10T07:44:36.342864+0000","last_fullsized":"2024-02-10T07:44:36.342864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:43:32.088479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339562+0000","last_change":"2024-02-10T07:44:11.537341+0000","last_active":"2024-02-10T07:44:36.339562+0000","last_peered":"2024-02-10T07:44:36.339562+0000","last_clean":"2024-02-10T07:44:36.339562+0000","last_became_active":"2024-02-10T07:44:11.536481+0000","last_became_peered":"2024-02-10T07:44:11.536481+0000","last_unstale":"2024-02-10T07:44:36.339562+0000","last_undegraded":"2024-02-10T07:44:36.339562+0000","last_fullsized":"2024-02-10T07:44:36.339562+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:57.374316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340165+0000","last_change":"2024-02-10T07:44:13.555875+0000","last_active":"2024-02-10T07:44:36.340165+0000","last_peered":"2024-02-10T07:44:36.340165+0000","last_clean":"2024-02-10T07:44:36.340165+0000","last_became_active":"2024-02-10T07:44:13.555353+0000","last_became_peered":"2024-02-10T07:44:13.555353+0000","last_unstale":"2024-02-10T07:44:36.340165+0000","last_undegraded":"2024-02-10T07:44:36.340165+0000","last_fullsized":"2024-02-10T07:44:36.340165+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:37:22.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341828+0000","last_change":"2024-02-10T07:44:07.511975+0000","last_active":"2024-02-10T07:44:36.341828+0000","last_peered":"2024-02-10T07:44:36.341828+0000","last_clean":"2024-02-10T07:44:36.341828+0000","last_became_active":"2024-02-10T07:44:07.511694+0000","last_became_peered":"2024-02-10T07:44:07.511694+0000","last_unstale":"2024-02-10T07:44:36.341828+0000","last_undegraded":"2024-02-10T07:44:36.341828+0000","last_fullsized":"2024-02-10T07:44:36.341828+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:39:41.454606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343054+0000","last_change":"2024-02-10T07:44:09.523985+0000","last_active":"2024-02-10T07:44:36.343054+0000","last_peered":"2024-02-10T07:44:36.343054+0000","last_clean":"2024-02-10T07:44:36.343054+0000","last_became_active":"2024-02-10T07:44:09.523529+0000","last_became_peered":"2024-02-10T07:44:09.523529+0000","last_unstale":"2024-02-10T07:44:36.343054+0000","last_undegraded":"2024-02-10T07:44:36.343054+0000","last_fullsized":"2024-02-10T07:44:36.343054+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:08.906179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340889+0000","last_change":"2024-02-10T07:44:11.538107+0000","last_active":"2024-02-10T07:44:36.340889+0000","last_peered":"2024-02-10T07:44:36.340889+0000","last_clean":"2024-02-10T07:44:36.340889+0000","last_became_active":"2024-02-10T07:44:11.537741+0000","last_became_peered":"2024-02-10T07:44:11.537741+0000","last_unstale":"2024-02-10T07:44:36.340889+0000","last_undegraded":"2024-02-10T07:44:36.340889+0000","last_fullsized":"2024-02-10T07:44:36.340889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:04:20.068785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411342,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738392,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.31},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.28899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.379},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.39800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T07:44:52.926 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph pg dump --format=json 2024-02-10T07:44:53.633 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:53 smithi022 ceph-mon[25704]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:53.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:53 smithi081 ceph-mon[30233]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:53.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:53 smithi027 ceph-mon[29844]: pgmap v11: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:54.250 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:55.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:55 smithi081 ceph-mon[30233]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:55.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:55 smithi027 ceph-mon[29844]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:55.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:55 smithi022 ceph-mon[25704]: pgmap v12: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:57.205 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:44:57.207 INFO:teuthology.orchestra.run.smithi022.stderr:dumped all 2024-02-10T07:44:57.444 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 podman[57809]: 2024-02-10 07:44:57.303893729 +0000 UTC m=+0.240782928 container create 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:57.697 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-mon[25704]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 podman[57809]: 2024-02-10 07:44:57.615019836 +0000 UTC m=+0.551909036 container init 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 podman[57809]: 2024-02-10 07:44:57.620572733 +0000 UTC m=+0.557461932 container start 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.632Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.632Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.633Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.22 port=9094 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.634Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.657Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.657Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.660Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-02-10T07:44:57.698 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:57.660Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-02-10T07:44:57.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:57 smithi081 ceph-mon[30233]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:57.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:57 smithi027 ceph-mon[29844]: pgmap v13: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:44:58.042 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 bash[57809]: 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b 2024-02-10T07:44:58.042 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:57 smithi022 systemd[1]: Started Ceph alertmanager.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE Bus STOPPING 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE Bus STOPPED 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE Bus STARTING 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE Serving on http://:::9283 2024-02-10T07:44:58.043 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:57 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:44:57] ENGINE Bus STARTED 2024-02-10T07:44:58.397 INFO:teuthology.orchestra.run.smithi022.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-02-10T07:44:56.352731+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595908,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":974,"num_read_kb":821,"num_write":697,"num_write_kb":4546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":673,"ondisk_log_size":673,"up":387,"acting":387,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":387,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":281223168,"kb_used":96620,"kb_used_data":3084,"kb_used_omap":4,"kb_used_meta":93499,"kb_avail":281126548,"statfs":{"total":287972524032,"available":287873585152,"internally_reserved":0,"allocated":3158016,"data_stored":2100345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4784,"internal_metadata":95743312},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002974"},"pg_stats":[{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341110+0000","last_change":"2024-02-10T07:44:13.557527+0000","last_active":"2024-02-10T07:44:36.341110+0000","last_peered":"2024-02-10T07:44:36.341110+0000","last_clean":"2024-02-10T07:44:36.341110+0000","last_became_active":"2024-02-10T07:44:13.556947+0000","last_became_peered":"2024-02-10T07:44:13.556947+0000","last_unstale":"2024-02-10T07:44:36.341110+0000","last_undegraded":"2024-02-10T07:44:36.341110+0000","last_fullsized":"2024-02-10T07:44:36.341110+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:13:46.282106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340176+0000","last_change":"2024-02-10T07:44:11.538312+0000","last_active":"2024-02-10T07:44:36.340176+0000","last_peered":"2024-02-10T07:44:36.340176+0000","last_clean":"2024-02-10T07:44:36.340176+0000","last_became_active":"2024-02-10T07:44:11.537998+0000","last_became_peered":"2024-02-10T07:44:11.537998+0000","last_unstale":"2024-02-10T07:44:36.340176+0000","last_undegraded":"2024-02-10T07:44:36.340176+0000","last_fullsized":"2024-02-10T07:44:36.340176+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:13:17.152364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342024+0000","last_change":"2024-02-10T07:44:07.511088+0000","last_active":"2024-02-10T07:44:36.342024+0000","last_peered":"2024-02-10T07:44:36.342024+0000","last_clean":"2024-02-10T07:44:36.342024+0000","last_became_active":"2024-02-10T07:44:07.510385+0000","last_became_peered":"2024-02-10T07:44:07.510385+0000","last_unstale":"2024-02-10T07:44:36.342024+0000","last_undegraded":"2024-02-10T07:44:36.342024+0000","last_fullsized":"2024-02-10T07:44:36.342024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:24:28.408798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341701+0000","last_change":"2024-02-10T07:44:09.527197+0000","last_active":"2024-02-10T07:44:36.341701+0000","last_peered":"2024-02-10T07:44:36.341701+0000","last_clean":"2024-02-10T07:44:36.341701+0000","last_became_active":"2024-02-10T07:44:09.526598+0000","last_became_peered":"2024-02-10T07:44:09.526598+0000","last_unstale":"2024-02-10T07:44:36.341701+0000","last_undegraded":"2024-02-10T07:44:36.341701+0000","last_fullsized":"2024-02-10T07:44:36.341701+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:43:11.136118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339392+0000","last_change":"2024-02-10T07:44:13.555219+0000","last_active":"2024-02-10T07:44:36.339392+0000","last_peered":"2024-02-10T07:44:36.339392+0000","last_clean":"2024-02-10T07:44:36.339392+0000","last_became_active":"2024-02-10T07:44:13.553853+0000","last_became_peered":"2024-02-10T07:44:13.553853+0000","last_unstale":"2024-02-10T07:44:36.339392+0000","last_undegraded":"2024-02-10T07:44:36.339392+0000","last_fullsized":"2024-02-10T07:44:36.339392+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:16:53.632025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340468+0000","last_change":"2024-02-10T07:44:11.543384+0000","last_active":"2024-02-10T07:44:36.340468+0000","last_peered":"2024-02-10T07:44:36.340468+0000","last_clean":"2024-02-10T07:44:36.340468+0000","last_became_active":"2024-02-10T07:44:11.542792+0000","last_became_peered":"2024-02-10T07:44:11.542792+0000","last_unstale":"2024-02-10T07:44:36.340468+0000","last_undegraded":"2024-02-10T07:44:36.340468+0000","last_fullsized":"2024-02-10T07:44:36.340468+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:11:32.529921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1e","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341991+0000","last_change":"2024-02-10T07:44:07.510076+0000","last_active":"2024-02-10T07:44:36.341991+0000","last_peered":"2024-02-10T07:44:36.341991+0000","last_clean":"2024-02-10T07:44:36.341991+0000","last_became_active":"2024-02-10T07:44:07.509467+0000","last_became_peered":"2024-02-10T07:44:07.509467+0000","last_unstale":"2024-02-10T07:44:36.341991+0000","last_undegraded":"2024-02-10T07:44:36.341991+0000","last_fullsized":"2024-02-10T07:44:36.341991+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:29:46.327660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341624+0000","last_change":"2024-02-10T07:44:09.526396+0000","last_active":"2024-02-10T07:44:36.341624+0000","last_peered":"2024-02-10T07:44:36.341624+0000","last_clean":"2024-02-10T07:44:36.341624+0000","last_became_active":"2024-02-10T07:44:09.526081+0000","last_became_peered":"2024-02-10T07:44:09.526081+0000","last_unstale":"2024-02-10T07:44:36.341624+0000","last_undegraded":"2024-02-10T07:44:36.341624+0000","last_fullsized":"2024-02-10T07:44:36.341624+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:02:58.093487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340238+0000","last_change":"2024-02-10T07:44:13.554626+0000","last_active":"2024-02-10T07:44:36.340238+0000","last_peered":"2024-02-10T07:44:36.340238+0000","last_clean":"2024-02-10T07:44:36.340238+0000","last_became_active":"2024-02-10T07:44:13.554344+0000","last_became_peered":"2024-02-10T07:44:13.554344+0000","last_unstale":"2024-02-10T07:44:36.340238+0000","last_undegraded":"2024-02-10T07:44:36.340238+0000","last_fullsized":"2024-02-10T07:44:36.340238+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:11:50.006023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341506+0000","last_change":"2024-02-10T07:44:11.541610+0000","last_active":"2024-02-10T07:44:36.341506+0000","last_peered":"2024-02-10T07:44:36.341506+0000","last_clean":"2024-02-10T07:44:36.341506+0000","last_became_active":"2024-02-10T07:44:11.541270+0000","last_became_peered":"2024-02-10T07:44:11.541270+0000","last_unstale":"2024-02-10T07:44:36.341506+0000","last_undegraded":"2024-02-10T07:44:36.341506+0000","last_fullsized":"2024-02-10T07:44:36.341506+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:20:57.155483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343371+0000","last_change":"2024-02-10T07:44:07.513280+0000","last_active":"2024-02-10T07:44:36.343371+0000","last_peered":"2024-02-10T07:44:36.343371+0000","last_clean":"2024-02-10T07:44:36.343371+0000","last_became_active":"2024-02-10T07:44:07.512394+0000","last_became_peered":"2024-02-10T07:44:07.512394+0000","last_unstale":"2024-02-10T07:44:36.343371+0000","last_undegraded":"2024-02-10T07:44:36.343371+0000","last_fullsized":"2024-02-10T07:44:36.343371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:21.926453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343033+0000","last_change":"2024-02-10T07:44:09.522685+0000","last_active":"2024-02-10T07:44:36.343033+0000","last_peered":"2024-02-10T07:44:36.343033+0000","last_clean":"2024-02-10T07:44:36.343033+0000","last_became_active":"2024-02-10T07:44:09.522010+0000","last_became_peered":"2024-02-10T07:44:09.522010+0000","last_unstale":"2024-02-10T07:44:36.343033+0000","last_undegraded":"2024-02-10T07:44:36.343033+0000","last_fullsized":"2024-02-10T07:44:36.343033+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:08:42.582421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1b","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340472+0000","last_change":"2024-02-10T07:44:13.554941+0000","last_active":"2024-02-10T07:44:36.340472+0000","last_peered":"2024-02-10T07:44:36.340472+0000","last_clean":"2024-02-10T07:44:36.340472+0000","last_became_active":"2024-02-10T07:44:13.554387+0000","last_became_peered":"2024-02-10T07:44:13.554387+0000","last_unstale":"2024-02-10T07:44:36.340472+0000","last_undegraded":"2024-02-10T07:44:36.340472+0000","last_fullsized":"2024-02-10T07:44:36.340472+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:37:28.332666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340582+0000","last_change":"2024-02-10T07:44:11.542774+0000","last_active":"2024-02-10T07:44:36.340582+0000","last_peered":"2024-02-10T07:44:36.340582+0000","last_clean":"2024-02-10T07:44:36.340582+0000","last_became_active":"2024-02-10T07:44:11.541837+0000","last_became_peered":"2024-02-10T07:44:11.541837+0000","last_unstale":"2024-02-10T07:44:36.340582+0000","last_undegraded":"2024-02-10T07:44:36.340582+0000","last_fullsized":"2024-02-10T07:44:36.340582+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:17:01.022750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341816+0000","last_change":"2024-02-10T07:44:07.510728+0000","last_active":"2024-02-10T07:44:36.341816+0000","last_peered":"2024-02-10T07:44:36.341816+0000","last_clean":"2024-02-10T07:44:36.341816+0000","last_became_active":"2024-02-10T07:44:07.510470+0000","last_became_peered":"2024-02-10T07:44:07.510470+0000","last_unstale":"2024-02-10T07:44:36.341816+0000","last_undegraded":"2024-02-10T07:44:36.341816+0000","last_fullsized":"2024-02-10T07:44:36.341816+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:19:30.919065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343326+0000","last_change":"2024-02-10T07:44:09.522627+0000","last_active":"2024-02-10T07:44:36.343326+0000","last_peered":"2024-02-10T07:44:36.343326+0000","last_clean":"2024-02-10T07:44:36.343326+0000","last_became_active":"2024-02-10T07:44:09.522043+0000","last_became_peered":"2024-02-10T07:44:09.522043+0000","last_unstale":"2024-02-10T07:44:36.343326+0000","last_undegraded":"2024-02-10T07:44:36.343326+0000","last_fullsized":"2024-02-10T07:44:36.343326+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:07:25.598568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1c","version":"30'1","reported_seq":16,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341866+0000","last_change":"2024-02-10T07:44:13.556884+0000","last_active":"2024-02-10T07:44:36.341866+0000","last_peered":"2024-02-10T07:44:36.341866+0000","last_clean":"2024-02-10T07:44:36.341866+0000","last_became_active":"2024-02-10T07:44:13.555485+0000","last_became_peered":"2024-02-10T07:44:13.555485+0000","last_unstale":"2024-02-10T07:44:36.341866+0000","last_undegraded":"2024-02-10T07:44:36.341866+0000","last_fullsized":"2024-02-10T07:44:36.341866+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:33:48.427105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342878+0000","last_change":"2024-02-10T07:44:11.541138+0000","last_active":"2024-02-10T07:44:36.342878+0000","last_peered":"2024-02-10T07:44:36.342878+0000","last_clean":"2024-02-10T07:44:36.342878+0000","last_became_active":"2024-02-10T07:44:11.540840+0000","last_became_peered":"2024-02-10T07:44:11.540840+0000","last_unstale":"2024-02-10T07:44:36.342878+0000","last_undegraded":"2024-02-10T07:44:36.342878+0000","last_fullsized":"2024-02-10T07:44:36.342878+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:46:51.081334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341796+0000","last_change":"2024-02-10T07:44:07.509381+0000","last_active":"2024-02-10T07:44:36.341796+0000","last_peered":"2024-02-10T07:44:36.341796+0000","last_clean":"2024-02-10T07:44:36.341796+0000","last_became_active":"2024-02-10T07:44:07.509073+0000","last_became_peered":"2024-02-10T07:44:07.509073+0000","last_unstale":"2024-02-10T07:44:36.341796+0000","last_undegraded":"2024-02-10T07:44:36.341796+0000","last_fullsized":"2024-02-10T07:44:36.341796+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:22:33.069618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340982+0000","last_change":"2024-02-10T07:44:09.523039+0000","last_active":"2024-02-10T07:44:36.340982+0000","last_peered":"2024-02-10T07:44:36.340982+0000","last_clean":"2024-02-10T07:44:36.340982+0000","last_became_active":"2024-02-10T07:44:09.522401+0000","last_became_peered":"2024-02-10T07:44:09.522401+0000","last_unstale":"2024-02-10T07:44:36.340982+0000","last_undegraded":"2024-02-10T07:44:36.340982+0000","last_fullsized":"2024-02-10T07:44:36.340982+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:11:08.947906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340799+0000","last_change":"2024-02-10T07:44:13.560016+0000","last_active":"2024-02-10T07:44:36.340799+0000","last_peered":"2024-02-10T07:44:36.340799+0000","last_clean":"2024-02-10T07:44:36.340799+0000","last_became_active":"2024-02-10T07:44:13.559681+0000","last_became_peered":"2024-02-10T07:44:13.559681+0000","last_unstale":"2024-02-10T07:44:36.340799+0000","last_undegraded":"2024-02-10T07:44:36.340799+0000","last_fullsized":"2024-02-10T07:44:36.340799+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:46:38.015414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341594+0000","last_change":"2024-02-10T07:44:11.537824+0000","last_active":"2024-02-10T07:44:36.341594+0000","last_peered":"2024-02-10T07:44:36.341594+0000","last_clean":"2024-02-10T07:44:36.341594+0000","last_became_active":"2024-02-10T07:44:11.537544+0000","last_became_peered":"2024-02-10T07:44:11.537544+0000","last_unstale":"2024-02-10T07:44:36.341594+0000","last_undegraded":"2024-02-10T07:44:36.341594+0000","last_fullsized":"2024-02-10T07:44:36.341594+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:12:03.769889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340851+0000","last_change":"2024-02-10T07:44:07.512551+0000","last_active":"2024-02-10T07:44:36.340851+0000","last_peered":"2024-02-10T07:44:36.340851+0000","last_clean":"2024-02-10T07:44:36.340851+0000","last_became_active":"2024-02-10T07:44:07.512253+0000","last_became_peered":"2024-02-10T07:44:07.512253+0000","last_unstale":"2024-02-10T07:44:36.340851+0000","last_undegraded":"2024-02-10T07:44:36.340851+0000","last_fullsized":"2024-02-10T07:44:36.340851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:10:02.236421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"30'5","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341509+0000","last_change":"2024-02-10T07:44:09.522161+0000","last_active":"2024-02-10T07:44:36.341509+0000","last_peered":"2024-02-10T07:44:36.341509+0000","last_clean":"2024-02-10T07:44:36.341509+0000","last_became_active":"2024-02-10T07:44:09.521867+0000","last_became_peered":"2024-02-10T07:44:09.521867+0000","last_unstale":"2024-02-10T07:44:36.341509+0000","last_undegraded":"2024-02-10T07:44:36.341509+0000","last_fullsized":"2024-02-10T07:44:36.341509+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:38:38.883409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341121+0000","last_change":"2024-02-10T07:44:13.555482+0000","last_active":"2024-02-10T07:44:36.341121+0000","last_peered":"2024-02-10T07:44:36.341121+0000","last_clean":"2024-02-10T07:44:36.341121+0000","last_became_active":"2024-02-10T07:44:13.553979+0000","last_became_peered":"2024-02-10T07:44:13.553979+0000","last_unstale":"2024-02-10T07:44:36.341121+0000","last_undegraded":"2024-02-10T07:44:36.341121+0000","last_fullsized":"2024-02-10T07:44:36.341121+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:24:48.122612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563663+0000","last_change":"2024-02-10T07:44:11.542004+0000","last_active":"2024-02-10T07:44:42.563663+0000","last_peered":"2024-02-10T07:44:42.563663+0000","last_clean":"2024-02-10T07:44:42.563663+0000","last_became_active":"2024-02-10T07:44:11.541147+0000","last_became_peered":"2024-02-10T07:44:11.541147+0000","last_unstale":"2024-02-10T07:44:42.563663+0000","last_undegraded":"2024-02-10T07:44:42.563663+0000","last_fullsized":"2024-02-10T07:44:42.563663+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:24:45.218856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.19","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339535+0000","last_change":"2024-02-10T07:44:07.511548+0000","last_active":"2024-02-10T07:44:36.339535+0000","last_peered":"2024-02-10T07:44:36.339535+0000","last_clean":"2024-02-10T07:44:36.339535+0000","last_became_active":"2024-02-10T07:44:07.510487+0000","last_became_peered":"2024-02-10T07:44:07.510487+0000","last_unstale":"2024-02-10T07:44:36.339535+0000","last_undegraded":"2024-02-10T07:44:36.339535+0000","last_fullsized":"2024-02-10T07:44:36.339535+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:27:17.147379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338674+0000","last_change":"2024-02-10T07:44:09.527066+0000","last_active":"2024-02-10T07:44:36.338674+0000","last_peered":"2024-02-10T07:44:36.338674+0000","last_clean":"2024-02-10T07:44:36.338674+0000","last_became_active":"2024-02-10T07:44:09.526764+0000","last_became_peered":"2024-02-10T07:44:09.526764+0000","last_unstale":"2024-02-10T07:44:36.338674+0000","last_undegraded":"2024-02-10T07:44:36.338674+0000","last_fullsized":"2024-02-10T07:44:36.338674+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:45:11.605353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341411+0000","last_change":"2024-02-10T07:44:13.556463+0000","last_active":"2024-02-10T07:44:36.341411+0000","last_peered":"2024-02-10T07:44:36.341411+0000","last_clean":"2024-02-10T07:44:36.341411+0000","last_became_active":"2024-02-10T07:44:13.555226+0000","last_became_peered":"2024-02-10T07:44:13.555226+0000","last_unstale":"2024-02-10T07:44:36.341411+0000","last_undegraded":"2024-02-10T07:44:36.341411+0000","last_fullsized":"2024-02-10T07:44:36.341411+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:08:32.539880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341284+0000","last_change":"2024-02-10T07:44:11.537841+0000","last_active":"2024-02-10T07:44:36.341284+0000","last_peered":"2024-02-10T07:44:36.341284+0000","last_clean":"2024-02-10T07:44:36.341284+0000","last_became_active":"2024-02-10T07:44:11.536933+0000","last_became_peered":"2024-02-10T07:44:11.536933+0000","last_unstale":"2024-02-10T07:44:36.341284+0000","last_undegraded":"2024-02-10T07:44:36.341284+0000","last_fullsized":"2024-02-10T07:44:36.341284+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:01:36.317990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340432+0000","last_change":"2024-02-10T07:44:07.513225+0000","last_active":"2024-02-10T07:44:36.340432+0000","last_peered":"2024-02-10T07:44:36.340432+0000","last_clean":"2024-02-10T07:44:36.340432+0000","last_became_active":"2024-02-10T07:44:07.512722+0000","last_became_peered":"2024-02-10T07:44:07.512722+0000","last_unstale":"2024-02-10T07:44:36.340432+0000","last_undegraded":"2024-02-10T07:44:36.340432+0000","last_fullsized":"2024-02-10T07:44:36.340432+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:35:33.410717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.19","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339092+0000","last_change":"2024-02-10T07:44:09.526189+0000","last_active":"2024-02-10T07:44:36.339092+0000","last_peered":"2024-02-10T07:44:36.339092+0000","last_clean":"2024-02-10T07:44:36.339092+0000","last_became_active":"2024-02-10T07:44:09.525787+0000","last_became_peered":"2024-02-10T07:44:09.525787+0000","last_unstale":"2024-02-10T07:44:36.339092+0000","last_undegraded":"2024-02-10T07:44:36.339092+0000","last_fullsized":"2024-02-10T07:44:36.339092+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:35:04.759257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341272+0000","last_change":"2024-02-10T07:44:13.556920+0000","last_active":"2024-02-10T07:44:36.341272+0000","last_peered":"2024-02-10T07:44:36.341272+0000","last_clean":"2024-02-10T07:44:36.341272+0000","last_became_active":"2024-02-10T07:44:13.556458+0000","last_became_peered":"2024-02-10T07:44:13.556458+0000","last_unstale":"2024-02-10T07:44:36.341272+0000","last_undegraded":"2024-02-10T07:44:36.341272+0000","last_fullsized":"2024-02-10T07:44:36.341272+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:09:02.836089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338923+0000","last_change":"2024-02-10T07:44:11.535402+0000","last_active":"2024-02-10T07:44:36.338923+0000","last_peered":"2024-02-10T07:44:36.338923+0000","last_clean":"2024-02-10T07:44:36.338923+0000","last_became_active":"2024-02-10T07:44:11.534703+0000","last_became_peered":"2024-02-10T07:44:11.534703+0000","last_unstale":"2024-02-10T07:44:36.338923+0000","last_undegraded":"2024-02-10T07:44:36.338923+0000","last_fullsized":"2024-02-10T07:44:36.338923+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:36:13.180093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342959+0000","last_change":"2024-02-10T07:44:07.514066+0000","last_active":"2024-02-10T07:44:36.342959+0000","last_peered":"2024-02-10T07:44:36.342959+0000","last_clean":"2024-02-10T07:44:36.342959+0000","last_became_active":"2024-02-10T07:44:07.513754+0000","last_became_peered":"2024-02-10T07:44:07.513754+0000","last_unstale":"2024-02-10T07:44:36.342959+0000","last_undegraded":"2024-02-10T07:44:36.342959+0000","last_fullsized":"2024-02-10T07:44:36.342959+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:06:04.576803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343543+0000","last_change":"2024-02-10T07:44:09.522767+0000","last_active":"2024-02-10T07:44:36.343543+0000","last_peered":"2024-02-10T07:44:36.343543+0000","last_clean":"2024-02-10T07:44:36.343543+0000","last_became_active":"2024-02-10T07:44:09.522208+0000","last_became_peered":"2024-02-10T07:44:09.522208+0000","last_unstale":"2024-02-10T07:44:36.343543+0000","last_undegraded":"2024-02-10T07:44:36.343543+0000","last_fullsized":"2024-02-10T07:44:36.343543+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:43:13.421982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340999+0000","last_change":"2024-02-10T07:44:13.555142+0000","last_active":"2024-02-10T07:44:36.340999+0000","last_peered":"2024-02-10T07:44:36.340999+0000","last_clean":"2024-02-10T07:44:36.340999+0000","last_became_active":"2024-02-10T07:44:13.554155+0000","last_became_peered":"2024-02-10T07:44:13.554155+0000","last_unstale":"2024-02-10T07:44:36.340999+0000","last_undegraded":"2024-02-10T07:44:36.340999+0000","last_fullsized":"2024-02-10T07:44:36.340999+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:15:55.124063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339024+0000","last_change":"2024-02-10T07:44:11.539614+0000","last_active":"2024-02-10T07:44:36.339024+0000","last_peered":"2024-02-10T07:44:36.339024+0000","last_clean":"2024-02-10T07:44:36.339024+0000","last_became_active":"2024-02-10T07:44:11.538208+0000","last_became_peered":"2024-02-10T07:44:11.538208+0000","last_unstale":"2024-02-10T07:44:36.339024+0000","last_undegraded":"2024-02-10T07:44:36.339024+0000","last_fullsized":"2024-02-10T07:44:36.339024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:23:06.358992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343392+0000","last_change":"2024-02-10T07:44:07.513398+0000","last_active":"2024-02-10T07:44:36.343392+0000","last_peered":"2024-02-10T07:44:36.343392+0000","last_clean":"2024-02-10T07:44:36.343392+0000","last_became_active":"2024-02-10T07:44:07.513127+0000","last_became_peered":"2024-02-10T07:44:07.513127+0000","last_unstale":"2024-02-10T07:44:36.343392+0000","last_undegraded":"2024-02-10T07:44:36.343392+0000","last_fullsized":"2024-02-10T07:44:36.343392+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:59:00.398184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.17","version":"30'6","reported_seq":32,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341809+0000","last_change":"2024-02-10T07:44:09.527791+0000","last_active":"2024-02-10T07:44:36.341809+0000","last_peered":"2024-02-10T07:44:36.341809+0000","last_clean":"2024-02-10T07:44:36.341809+0000","last_became_active":"2024-02-10T07:44:09.527510+0000","last_became_peered":"2024-02-10T07:44:09.527510+0000","last_unstale":"2024-02-10T07:44:36.341809+0000","last_undegraded":"2024-02-10T07:44:36.341809+0000","last_fullsized":"2024-02-10T07:44:36.341809+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:26:21.793062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340640+0000","last_change":"2024-02-10T07:44:13.554681+0000","last_active":"2024-02-10T07:44:36.340640+0000","last_peered":"2024-02-10T07:44:36.340640+0000","last_clean":"2024-02-10T07:44:36.340640+0000","last_became_active":"2024-02-10T07:44:13.554408+0000","last_became_peered":"2024-02-10T07:44:13.554408+0000","last_unstale":"2024-02-10T07:44:36.340640+0000","last_undegraded":"2024-02-10T07:44:36.340640+0000","last_fullsized":"2024-02-10T07:44:36.340640+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:40:19.737670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341679+0000","last_change":"2024-02-10T07:44:11.543485+0000","last_active":"2024-02-10T07:44:36.341679+0000","last_peered":"2024-02-10T07:44:36.341679+0000","last_clean":"2024-02-10T07:44:36.341679+0000","last_became_active":"2024-02-10T07:44:11.543132+0000","last_became_peered":"2024-02-10T07:44:11.543132+0000","last_unstale":"2024-02-10T07:44:36.341679+0000","last_undegraded":"2024-02-10T07:44:36.341679+0000","last_fullsized":"2024-02-10T07:44:36.341679+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:16:12.458455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343267+0000","last_change":"2024-02-10T07:44:07.513177+0000","last_active":"2024-02-10T07:44:36.343267+0000","last_peered":"2024-02-10T07:44:36.343267+0000","last_clean":"2024-02-10T07:44:36.343267+0000","last_became_active":"2024-02-10T07:44:07.512270+0000","last_became_peered":"2024-02-10T07:44:07.512270+0000","last_unstale":"2024-02-10T07:44:36.343267+0000","last_undegraded":"2024-02-10T07:44:36.343267+0000","last_fullsized":"2024-02-10T07:44:36.343267+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:00:23.656408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342999+0000","last_change":"2024-02-10T07:44:09.523182+0000","last_active":"2024-02-10T07:44:36.342999+0000","last_peered":"2024-02-10T07:44:36.342999+0000","last_clean":"2024-02-10T07:44:36.342999+0000","last_became_active":"2024-02-10T07:44:09.522189+0000","last_became_peered":"2024-02-10T07:44:09.522189+0000","last_unstale":"2024-02-10T07:44:36.342999+0000","last_undegraded":"2024-02-10T07:44:36.342999+0000","last_fullsized":"2024-02-10T07:44:36.342999+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:57:12.167059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340511+0000","last_change":"2024-02-10T07:44:13.554804+0000","last_active":"2024-02-10T07:44:36.340511+0000","last_peered":"2024-02-10T07:44:36.340511+0000","last_clean":"2024-02-10T07:44:36.340511+0000","last_became_active":"2024-02-10T07:44:13.554446+0000","last_became_peered":"2024-02-10T07:44:13.554446+0000","last_unstale":"2024-02-10T07:44:36.340511+0000","last_undegraded":"2024-02-10T07:44:36.340511+0000","last_fullsized":"2024-02-10T07:44:36.340511+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:01:26.831537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339390+0000","last_change":"2024-02-10T07:44:11.536996+0000","last_active":"2024-02-10T07:44:36.339390+0000","last_peered":"2024-02-10T07:44:36.339390+0000","last_clean":"2024-02-10T07:44:36.339390+0000","last_became_active":"2024-02-10T07:44:11.536126+0000","last_became_peered":"2024-02-10T07:44:11.536126+0000","last_unstale":"2024-02-10T07:44:36.339390+0000","last_undegraded":"2024-02-10T07:44:36.339390+0000","last_fullsized":"2024-02-10T07:44:36.339390+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:19:57.891667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342763+0000","last_change":"2024-02-10T07:44:07.512872+0000","last_active":"2024-02-10T07:44:36.342763+0000","last_peered":"2024-02-10T07:44:36.342763+0000","last_clean":"2024-02-10T07:44:36.342763+0000","last_became_active":"2024-02-10T07:44:07.512560+0000","last_became_peered":"2024-02-10T07:44:07.512560+0000","last_unstale":"2024-02-10T07:44:36.342763+0000","last_undegraded":"2024-02-10T07:44:36.342763+0000","last_fullsized":"2024-02-10T07:44:36.342763+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:35:39.277816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343082+0000","last_change":"2024-02-10T07:44:09.522255+0000","last_active":"2024-02-10T07:44:36.343082+0000","last_peered":"2024-02-10T07:44:36.343082+0000","last_clean":"2024-02-10T07:44:36.343082+0000","last_became_active":"2024-02-10T07:44:09.521848+0000","last_became_peered":"2024-02-10T07:44:09.521848+0000","last_unstale":"2024-02-10T07:44:36.343082+0000","last_undegraded":"2024-02-10T07:44:36.343082+0000","last_fullsized":"2024-02-10T07:44:36.343082+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:35:07.033421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340974+0000","last_change":"2024-02-10T07:44:13.555337+0000","last_active":"2024-02-10T07:44:36.340974+0000","last_peered":"2024-02-10T07:44:36.340974+0000","last_clean":"2024-02-10T07:44:36.340974+0000","last_became_active":"2024-02-10T07:44:13.553637+0000","last_became_peered":"2024-02-10T07:44:13.553637+0000","last_unstale":"2024-02-10T07:44:36.340974+0000","last_undegraded":"2024-02-10T07:44:36.340974+0000","last_fullsized":"2024-02-10T07:44:36.340974+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:09:12.367800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563667+0000","last_change":"2024-02-10T07:44:11.538707+0000","last_active":"2024-02-10T07:44:42.563667+0000","last_peered":"2024-02-10T07:44:42.563667+0000","last_clean":"2024-02-10T07:44:42.563667+0000","last_became_active":"2024-02-10T07:44:11.538414+0000","last_became_peered":"2024-02-10T07:44:11.538414+0000","last_unstale":"2024-02-10T07:44:42.563667+0000","last_undegraded":"2024-02-10T07:44:42.563667+0000","last_fullsized":"2024-02-10T07:44:42.563667+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:54:13.722018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341304+0000","last_change":"2024-02-10T07:44:07.512731+0000","last_active":"2024-02-10T07:44:36.341304+0000","last_peered":"2024-02-10T07:44:36.341304+0000","last_clean":"2024-02-10T07:44:36.341304+0000","last_became_active":"2024-02-10T07:44:07.512338+0000","last_became_peered":"2024-02-10T07:44:07.512338+0000","last_unstale":"2024-02-10T07:44:36.341304+0000","last_undegraded":"2024-02-10T07:44:36.341304+0000","last_fullsized":"2024-02-10T07:44:36.341304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:11:50.141235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339744+0000","last_change":"2024-02-10T07:44:09.524972+0000","last_active":"2024-02-10T07:44:36.339744+0000","last_peered":"2024-02-10T07:44:36.339744+0000","last_clean":"2024-02-10T07:44:36.339744+0000","last_became_active":"2024-02-10T07:44:09.524683+0000","last_became_peered":"2024-02-10T07:44:09.524683+0000","last_unstale":"2024-02-10T07:44:36.339744+0000","last_undegraded":"2024-02-10T07:44:36.339744+0000","last_fullsized":"2024-02-10T07:44:36.339744+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:39:02.389958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340935+0000","last_change":"2024-02-10T07:44:13.555129+0000","last_active":"2024-02-10T07:44:36.340935+0000","last_peered":"2024-02-10T07:44:36.340935+0000","last_clean":"2024-02-10T07:44:36.340935+0000","last_became_active":"2024-02-10T07:44:13.554648+0000","last_became_peered":"2024-02-10T07:44:13.554648+0000","last_unstale":"2024-02-10T07:44:36.340935+0000","last_undegraded":"2024-02-10T07:44:36.340935+0000","last_fullsized":"2024-02-10T07:44:36.340935+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:35:42.035642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563684+0000","last_change":"2024-02-10T07:44:11.538072+0000","last_active":"2024-02-10T07:44:42.563684+0000","last_peered":"2024-02-10T07:44:42.563684+0000","last_clean":"2024-02-10T07:44:42.563684+0000","last_became_active":"2024-02-10T07:44:11.537669+0000","last_became_peered":"2024-02-10T07:44:11.537669+0000","last_unstale":"2024-02-10T07:44:42.563684+0000","last_undegraded":"2024-02-10T07:44:42.563684+0000","last_fullsized":"2024-02-10T07:44:42.563684+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:54:42.820531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340747+0000","last_change":"2024-02-10T07:44:07.510570+0000","last_active":"2024-02-10T07:44:36.340747+0000","last_peered":"2024-02-10T07:44:36.340747+0000","last_clean":"2024-02-10T07:44:36.340747+0000","last_became_active":"2024-02-10T07:44:07.510245+0000","last_became_peered":"2024-02-10T07:44:07.510245+0000","last_unstale":"2024-02-10T07:44:36.340747+0000","last_undegraded":"2024-02-10T07:44:36.340747+0000","last_fullsized":"2024-02-10T07:44:36.340747+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:53:25.229381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.13","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342759+0000","last_change":"2024-02-10T07:44:09.520674+0000","last_active":"2024-02-10T07:44:36.342759+0000","last_peered":"2024-02-10T07:44:36.342759+0000","last_clean":"2024-02-10T07:44:36.342759+0000","last_became_active":"2024-02-10T07:44:09.520360+0000","last_became_peered":"2024-02-10T07:44:09.520360+0000","last_unstale":"2024-02-10T07:44:36.342759+0000","last_undegraded":"2024-02-10T07:44:36.342759+0000","last_fullsized":"2024-02-10T07:44:36.342759+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:08:35.506767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342733+0000","last_change":"2024-02-10T07:44:13.556851+0000","last_active":"2024-02-10T07:44:36.342733+0000","last_peered":"2024-02-10T07:44:36.342733+0000","last_clean":"2024-02-10T07:44:36.342733+0000","last_became_active":"2024-02-10T07:44:13.556481+0000","last_became_peered":"2024-02-10T07:44:13.556481+0000","last_unstale":"2024-02-10T07:44:36.342733+0000","last_undegraded":"2024-02-10T07:44:36.342733+0000","last_fullsized":"2024-02-10T07:44:36.342733+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:48:33.112634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341445+0000","last_change":"2024-02-10T07:44:11.538136+0000","last_active":"2024-02-10T07:44:36.341445+0000","last_peered":"2024-02-10T07:44:36.341445+0000","last_clean":"2024-02-10T07:44:36.341445+0000","last_became_active":"2024-02-10T07:44:11.537683+0000","last_became_peered":"2024-02-10T07:44:11.537683+0000","last_unstale":"2024-02-10T07:44:36.341445+0000","last_undegraded":"2024-02-10T07:44:36.341445+0000","last_fullsized":"2024-02-10T07:44:36.341445+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:44:20.746410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340696+0000","last_change":"2024-02-10T07:44:07.514186+0000","last_active":"2024-02-10T07:44:36.340696+0000","last_peered":"2024-02-10T07:44:36.340696+0000","last_clean":"2024-02-10T07:44:36.340696+0000","last_became_active":"2024-02-10T07:44:07.513907+0000","last_became_peered":"2024-02-10T07:44:07.513907+0000","last_unstale":"2024-02-10T07:44:36.340696+0000","last_undegraded":"2024-02-10T07:44:36.340696+0000","last_fullsized":"2024-02-10T07:44:36.340696+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:03:33.682218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.10","version":"30'4","reported_seq":29,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340064+0000","last_change":"2024-02-10T07:44:09.524592+0000","last_active":"2024-02-10T07:44:36.340064+0000","last_peered":"2024-02-10T07:44:36.340064+0000","last_clean":"2024-02-10T07:44:36.340064+0000","last_became_active":"2024-02-10T07:44:09.524351+0000","last_became_peered":"2024-02-10T07:44:09.524351+0000","last_unstale":"2024-02-10T07:44:36.340064+0000","last_undegraded":"2024-02-10T07:44:36.340064+0000","last_fullsized":"2024-02-10T07:44:36.340064+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:36:57.290271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341189+0000","last_change":"2024-02-10T07:44:13.560537+0000","last_active":"2024-02-10T07:44:36.341189+0000","last_peered":"2024-02-10T07:44:36.341189+0000","last_clean":"2024-02-10T07:44:36.341189+0000","last_became_active":"2024-02-10T07:44:13.560134+0000","last_became_peered":"2024-02-10T07:44:13.560134+0000","last_unstale":"2024-02-10T07:44:36.341189+0000","last_undegraded":"2024-02-10T07:44:36.341189+0000","last_fullsized":"2024-02-10T07:44:36.341189+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:22:57.271941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341371+0000","last_change":"2024-02-10T07:44:11.535946+0000","last_active":"2024-02-10T07:44:36.341371+0000","last_peered":"2024-02-10T07:44:36.341371+0000","last_clean":"2024-02-10T07:44:36.341371+0000","last_became_active":"2024-02-10T07:44:11.535603+0000","last_became_peered":"2024-02-10T07:44:11.535603+0000","last_unstale":"2024-02-10T07:44:36.341371+0000","last_undegraded":"2024-02-10T07:44:36.341371+0000","last_fullsized":"2024-02-10T07:44:36.341371+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:17:49.836513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340609+0000","last_change":"2024-02-10T07:44:07.513566+0000","last_active":"2024-02-10T07:44:36.340609+0000","last_peered":"2024-02-10T07:44:36.340609+0000","last_clean":"2024-02-10T07:44:36.340609+0000","last_became_active":"2024-02-10T07:44:07.513283+0000","last_became_peered":"2024-02-10T07:44:07.513283+0000","last_unstale":"2024-02-10T07:44:36.340609+0000","last_undegraded":"2024-02-10T07:44:36.340609+0000","last_fullsized":"2024-02-10T07:44:36.340609+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:08:04.575929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340502+0000","last_change":"2024-02-10T07:44:09.521875+0000","last_active":"2024-02-10T07:44:36.340502+0000","last_peered":"2024-02-10T07:44:36.340502+0000","last_clean":"2024-02-10T07:44:36.340502+0000","last_became_active":"2024-02-10T07:44:09.521527+0000","last_became_peered":"2024-02-10T07:44:09.521527+0000","last_unstale":"2024-02-10T07:44:36.340502+0000","last_undegraded":"2024-02-10T07:44:36.340502+0000","last_fullsized":"2024-02-10T07:44:36.340502+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:22:02.642569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342120+0000","last_change":"2024-02-10T07:44:13.556643+0000","last_active":"2024-02-10T07:44:36.342120+0000","last_peered":"2024-02-10T07:44:36.342120+0000","last_clean":"2024-02-10T07:44:36.342120+0000","last_became_active":"2024-02-10T07:44:13.556356+0000","last_became_peered":"2024-02-10T07:44:13.556356+0000","last_unstale":"2024-02-10T07:44:36.342120+0000","last_undegraded":"2024-02-10T07:44:36.342120+0000","last_fullsized":"2024-02-10T07:44:36.342120+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:31:01.042207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"23'2","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343091+0000","last_change":"2024-02-10T07:44:07.510878+0000","last_active":"2024-02-10T07:44:36.343091+0000","last_peered":"2024-02-10T07:44:36.343091+0000","last_clean":"2024-02-10T07:44:36.343091+0000","last_became_active":"2024-02-10T07:44:07.510596+0000","last_became_peered":"2024-02-10T07:44:07.510596+0000","last_unstale":"2024-02-10T07:44:36.343091+0000","last_undegraded":"2024-02-10T07:44:36.343091+0000","last_fullsized":"2024-02-10T07:44:36.343091+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:03:56.341218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"30'11","reported_seq":42,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342487+0000","last_change":"2024-02-10T07:44:09.525951+0000","last_active":"2024-02-10T07:44:36.342487+0000","last_peered":"2024-02-10T07:44:36.342487+0000","last_clean":"2024-02-10T07:44:36.342487+0000","last_became_active":"2024-02-10T07:44:09.525725+0000","last_became_peered":"2024-02-10T07:44:09.525725+0000","last_unstale":"2024-02-10T07:44:36.342487+0000","last_undegraded":"2024-02-10T07:44:36.342487+0000","last_fullsized":"2024-02-10T07:44:36.342487+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:13.837684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563769+0000","last_change":"2024-02-10T07:44:11.538707+0000","last_active":"2024-02-10T07:44:42.563769+0000","last_peered":"2024-02-10T07:44:42.563769+0000","last_clean":"2024-02-10T07:44:42.563769+0000","last_became_active":"2024-02-10T07:44:11.538433+0000","last_became_peered":"2024-02-10T07:44:11.538433+0000","last_unstale":"2024-02-10T07:44:42.563769+0000","last_undegraded":"2024-02-10T07:44:42.563769+0000","last_fullsized":"2024-02-10T07:44:42.563769+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:47:49.712669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339299+0000","last_change":"2024-02-10T07:44:13.555289+0000","last_active":"2024-02-10T07:44:36.339299+0000","last_peered":"2024-02-10T07:44:36.339299+0000","last_clean":"2024-02-10T07:44:36.339299+0000","last_became_active":"2024-02-10T07:44:13.554789+0000","last_became_peered":"2024-02-10T07:44:13.554789+0000","last_unstale":"2024-02-10T07:44:36.339299+0000","last_undegraded":"2024-02-10T07:44:36.339299+0000","last_fullsized":"2024-02-10T07:44:36.339299+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:02:11.835869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341244+0000","last_change":"2024-02-10T07:44:07.511613+0000","last_active":"2024-02-10T07:44:36.341244+0000","last_peered":"2024-02-10T07:44:36.341244+0000","last_clean":"2024-02-10T07:44:36.341244+0000","last_became_active":"2024-02-10T07:44:07.510220+0000","last_became_peered":"2024-02-10T07:44:07.510220+0000","last_unstale":"2024-02-10T07:44:36.341244+0000","last_undegraded":"2024-02-10T07:44:36.341244+0000","last_fullsized":"2024-02-10T07:44:36.341244+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:34:50.227635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"30'16","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563640+0000","last_change":"2024-02-10T07:44:09.520915+0000","last_active":"2024-02-10T07:44:42.563640+0000","last_peered":"2024-02-10T07:44:42.563640+0000","last_clean":"2024-02-10T07:44:42.563640+0000","last_became_active":"2024-02-10T07:44:09.520587+0000","last_became_peered":"2024-02-10T07:44:09.520587+0000","last_unstale":"2024-02-10T07:44:42.563640+0000","last_undegraded":"2024-02-10T07:44:42.563640+0000","last_fullsized":"2024-02-10T07:44:42.563640+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:50:49.331970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340508+0000","last_change":"2024-02-10T07:44:11.537804+0000","last_active":"2024-02-10T07:44:36.340508+0000","last_peered":"2024-02-10T07:44:36.340508+0000","last_clean":"2024-02-10T07:44:36.340508+0000","last_became_active":"2024-02-10T07:44:11.537443+0000","last_became_peered":"2024-02-10T07:44:11.537443+0000","last_unstale":"2024-02-10T07:44:36.340508+0000","last_undegraded":"2024-02-10T07:44:36.340508+0000","last_fullsized":"2024-02-10T07:44:36.340508+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:21:25.733201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340398+0000","last_change":"2024-02-10T07:44:13.553313+0000","last_active":"2024-02-10T07:44:36.340398+0000","last_peered":"2024-02-10T07:44:36.340398+0000","last_clean":"2024-02-10T07:44:36.340398+0000","last_became_active":"2024-02-10T07:44:13.552865+0000","last_became_peered":"2024-02-10T07:44:13.552865+0000","last_unstale":"2024-02-10T07:44:36.340398+0000","last_undegraded":"2024-02-10T07:44:36.340398+0000","last_fullsized":"2024-02-10T07:44:36.340398+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:29:54.822325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341017+0000","last_change":"2024-02-10T07:44:07.513499+0000","last_active":"2024-02-10T07:44:36.341017+0000","last_peered":"2024-02-10T07:44:36.341017+0000","last_clean":"2024-02-10T07:44:36.341017+0000","last_became_active":"2024-02-10T07:44:07.513238+0000","last_became_peered":"2024-02-10T07:44:07.513238+0000","last_unstale":"2024-02-10T07:44:36.341017+0000","last_undegraded":"2024-02-10T07:44:36.341017+0000","last_fullsized":"2024-02-10T07:44:36.341017+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:34:09.731229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"30'19","reported_seq":59,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342904+0000","last_change":"2024-02-10T07:44:09.520474+0000","last_active":"2024-02-10T07:44:36.342904+0000","last_peered":"2024-02-10T07:44:36.342904+0000","last_clean":"2024-02-10T07:44:36.342904+0000","last_became_active":"2024-02-10T07:44:09.520077+0000","last_became_peered":"2024-02-10T07:44:09.520077+0000","last_unstale":"2024-02-10T07:44:36.342904+0000","last_undegraded":"2024-02-10T07:44:36.342904+0000","last_fullsized":"2024-02-10T07:44:36.342904+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:38:39.985450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341063+0000","last_change":"2024-02-10T07:44:11.537700+0000","last_active":"2024-02-10T07:44:36.341063+0000","last_peered":"2024-02-10T07:44:36.341063+0000","last_clean":"2024-02-10T07:44:36.341063+0000","last_became_active":"2024-02-10T07:44:11.537339+0000","last_became_peered":"2024-02-10T07:44:11.537339+0000","last_unstale":"2024-02-10T07:44:36.341063+0000","last_undegraded":"2024-02-10T07:44:36.341063+0000","last_fullsized":"2024-02-10T07:44:36.341063+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:11:59.196770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342257+0000","last_change":"2024-02-10T07:44:13.560152+0000","last_active":"2024-02-10T07:44:36.342257+0000","last_peered":"2024-02-10T07:44:36.342257+0000","last_clean":"2024-02-10T07:44:36.342257+0000","last_became_active":"2024-02-10T07:44:13.559756+0000","last_became_peered":"2024-02-10T07:44:13.559756+0000","last_unstale":"2024-02-10T07:44:36.342257+0000","last_undegraded":"2024-02-10T07:44:36.342257+0000","last_fullsized":"2024-02-10T07:44:36.342257+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:22:30.661298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"30'18","reported_seq":55,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340087+0000","last_change":"2024-02-10T07:44:09.523085+0000","last_active":"2024-02-10T07:44:36.340087+0000","last_peered":"2024-02-10T07:44:36.340087+0000","last_clean":"2024-02-10T07:44:36.340087+0000","last_became_active":"2024-02-10T07:44:09.522442+0000","last_became_peered":"2024-02-10T07:44:09.522442+0000","last_unstale":"2024-02-10T07:44:36.340087+0000","last_undegraded":"2024-02-10T07:44:36.340087+0000","last_fullsized":"2024-02-10T07:44:36.340087+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:18:55.612922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339304+0000","last_change":"2024-02-10T07:44:07.512395+0000","last_active":"2024-02-10T07:44:36.339304+0000","last_peered":"2024-02-10T07:44:36.339304+0000","last_clean":"2024-02-10T07:44:36.339304+0000","last_became_active":"2024-02-10T07:44:07.511570+0000","last_became_peered":"2024-02-10T07:44:07.511570+0000","last_unstale":"2024-02-10T07:44:36.339304+0000","last_undegraded":"2024-02-10T07:44:36.339304+0000","last_fullsized":"2024-02-10T07:44:36.339304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:32:05.713726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341733+0000","last_change":"2024-02-10T07:44:11.539349+0000","last_active":"2024-02-10T07:44:36.341733+0000","last_peered":"2024-02-10T07:44:36.341733+0000","last_clean":"2024-02-10T07:44:36.341733+0000","last_became_active":"2024-02-10T07:44:11.537888+0000","last_became_peered":"2024-02-10T07:44:11.537888+0000","last_unstale":"2024-02-10T07:44:36.341733+0000","last_undegraded":"2024-02-10T07:44:36.341733+0000","last_fullsized":"2024-02-10T07:44:36.341733+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:12:57.192641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341100+0000","last_change":"2024-02-10T07:44:13.555785+0000","last_active":"2024-02-10T07:44:36.341100+0000","last_peered":"2024-02-10T07:44:36.341100+0000","last_clean":"2024-02-10T07:44:36.341100+0000","last_became_active":"2024-02-10T07:44:13.555176+0000","last_became_peered":"2024-02-10T07:44:13.555176+0000","last_unstale":"2024-02-10T07:44:36.341100+0000","last_undegraded":"2024-02-10T07:44:36.341100+0000","last_fullsized":"2024-02-10T07:44:36.341100+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:12:59.832906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339550+0000","last_change":"2024-02-10T07:44:09.527329+0000","last_active":"2024-02-10T07:44:36.339550+0000","last_peered":"2024-02-10T07:44:36.339550+0000","last_clean":"2024-02-10T07:44:36.339550+0000","last_became_active":"2024-02-10T07:44:09.526246+0000","last_became_peered":"2024-02-10T07:44:09.526246+0000","last_unstale":"2024-02-10T07:44:36.339550+0000","last_undegraded":"2024-02-10T07:44:36.339550+0000","last_fullsized":"2024-02-10T07:44:36.339550+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:00:22.023550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341936+0000","last_change":"2024-02-10T07:44:07.512537+0000","last_active":"2024-02-10T07:44:36.341936+0000","last_peered":"2024-02-10T07:44:36.341936+0000","last_clean":"2024-02-10T07:44:36.341936+0000","last_became_active":"2024-02-10T07:44:07.512249+0000","last_became_peered":"2024-02-10T07:44:07.512249+0000","last_unstale":"2024-02-10T07:44:36.341936+0000","last_undegraded":"2024-02-10T07:44:36.341936+0000","last_fullsized":"2024-02-10T07:44:36.341936+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:13.533086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"31'208","reported_seq":314,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:38.438336+0000","last_change":"2024-02-10T07:43:55.035647+0000","last_active":"2024-02-10T07:44:38.438336+0000","last_peered":"2024-02-10T07:44:38.438336+0000","last_clean":"2024-02-10T07:44:38.438336+0000","last_became_active":"2024-02-10T07:43:55.035360+0000","last_became_peered":"2024-02-10T07:43:55.035360+0000","last_unstale":"2024-02-10T07:44:38.438336+0000","last_undegraded":"2024-02-10T07:44:38.438336+0000","last_fullsized":"2024-02-10T07:44:38.438336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:43:54.028946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:43:54.028946+0000","last_clean_scrub_stamp":"2024-02-10T07:43:54.028946+0000","objects_scrubbed":0,"log_size":208,"log_dups_size":0,"ondisk_log_size":208,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:34:18.898006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341385+0000","last_change":"2024-02-10T07:44:11.543205+0000","last_active":"2024-02-10T07:44:36.341385+0000","last_peered":"2024-02-10T07:44:36.341385+0000","last_clean":"2024-02-10T07:44:36.341385+0000","last_became_active":"2024-02-10T07:44:11.542603+0000","last_became_peered":"2024-02-10T07:44:11.542603+0000","last_unstale":"2024-02-10T07:44:36.341385+0000","last_undegraded":"2024-02-10T07:44:36.341385+0000","last_fullsized":"2024-02-10T07:44:36.341385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:24:56.688038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341668+0000","last_change":"2024-02-10T07:44:13.555726+0000","last_active":"2024-02-10T07:44:36.341668+0000","last_peered":"2024-02-10T07:44:36.341668+0000","last_clean":"2024-02-10T07:44:36.341668+0000","last_became_active":"2024-02-10T07:44:13.554674+0000","last_became_peered":"2024-02-10T07:44:13.554674+0000","last_unstale":"2024-02-10T07:44:36.341668+0000","last_undegraded":"2024-02-10T07:44:36.341668+0000","last_fullsized":"2024-02-10T07:44:36.341668+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:40:34.479843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"30'14","reported_seq":44,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339559+0000","last_change":"2024-02-10T07:44:09.526625+0000","last_active":"2024-02-10T07:44:36.339559+0000","last_peered":"2024-02-10T07:44:36.339559+0000","last_clean":"2024-02-10T07:44:36.339559+0000","last_became_active":"2024-02-10T07:44:09.526250+0000","last_became_peered":"2024-02-10T07:44:09.526250+0000","last_unstale":"2024-02-10T07:44:36.339559+0000","last_undegraded":"2024-02-10T07:44:36.339559+0000","last_fullsized":"2024-02-10T07:44:36.339559+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:29:24.829423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'1","reported_seq":33,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340075+0000","last_change":"2024-02-10T07:44:07.512818+0000","last_active":"2024-02-10T07:44:36.340075+0000","last_peered":"2024-02-10T07:44:36.340075+0000","last_clean":"2024-02-10T07:44:36.340075+0000","last_became_active":"2024-02-10T07:44:07.512486+0000","last_became_peered":"2024-02-10T07:44:07.512486+0000","last_unstale":"2024-02-10T07:44:36.340075+0000","last_undegraded":"2024-02-10T07:44:36.340075+0000","last_fullsized":"2024-02-10T07:44:36.340075+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:14:09.343865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341418+0000","last_change":"2024-02-10T07:44:11.541376+0000","last_active":"2024-02-10T07:44:36.341418+0000","last_peered":"2024-02-10T07:44:36.341418+0000","last_clean":"2024-02-10T07:44:36.341418+0000","last_became_active":"2024-02-10T07:44:11.541061+0000","last_became_peered":"2024-02-10T07:44:11.541061+0000","last_unstale":"2024-02-10T07:44:36.341418+0000","last_undegraded":"2024-02-10T07:44:36.341418+0000","last_fullsized":"2024-02-10T07:44:36.341418+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:46:04.124370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340748+0000","last_change":"2024-02-10T07:44:13.554869+0000","last_active":"2024-02-10T07:44:36.340748+0000","last_peered":"2024-02-10T07:44:36.340748+0000","last_clean":"2024-02-10T07:44:36.340748+0000","last_became_active":"2024-02-10T07:44:13.554481+0000","last_became_peered":"2024-02-10T07:44:13.554481+0000","last_unstale":"2024-02-10T07:44:36.340748+0000","last_undegraded":"2024-02-10T07:44:36.340748+0000","last_fullsized":"2024-02-10T07:44:36.340748+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:25:33.264675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343222+0000","last_change":"2024-02-10T07:44:07.513599+0000","last_active":"2024-02-10T07:44:36.343222+0000","last_peered":"2024-02-10T07:44:36.343222+0000","last_clean":"2024-02-10T07:44:36.343222+0000","last_became_active":"2024-02-10T07:44:07.512707+0000","last_became_peered":"2024-02-10T07:44:07.512707+0000","last_unstale":"2024-02-10T07:44:36.343222+0000","last_undegraded":"2024-02-10T07:44:36.343222+0000","last_fullsized":"2024-02-10T07:44:36.343222+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:39:26.996006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.4","version":"30'28","reported_seq":80,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563355+0000","last_change":"2024-02-10T07:44:09.526443+0000","last_active":"2024-02-10T07:44:42.563355+0000","last_peered":"2024-02-10T07:44:42.563355+0000","last_clean":"2024-02-10T07:44:42.563355+0000","last_became_active":"2024-02-10T07:44:09.526059+0000","last_became_peered":"2024-02-10T07:44:09.526059+0000","last_unstale":"2024-02-10T07:44:42.563355+0000","last_undegraded":"2024-02-10T07:44:42.563355+0000","last_fullsized":"2024-02-10T07:44:42.563355+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:09:15.908095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563642+0000","last_change":"2024-02-10T07:44:11.537809+0000","last_active":"2024-02-10T07:44:42.563642+0000","last_peered":"2024-02-10T07:44:42.563642+0000","last_clean":"2024-02-10T07:44:42.563642+0000","last_became_active":"2024-02-10T07:44:11.537496+0000","last_became_peered":"2024-02-10T07:44:11.537496+0000","last_unstale":"2024-02-10T07:44:42.563642+0000","last_undegraded":"2024-02-10T07:44:42.563642+0000","last_fullsized":"2024-02-10T07:44:42.563642+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:59:02.165808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340322+0000","last_change":"2024-02-10T07:44:13.554740+0000","last_active":"2024-02-10T07:44:36.340322+0000","last_peered":"2024-02-10T07:44:36.340322+0000","last_clean":"2024-02-10T07:44:36.340322+0000","last_became_active":"2024-02-10T07:44:13.554094+0000","last_became_peered":"2024-02-10T07:44:13.554094+0000","last_unstale":"2024-02-10T07:44:36.340322+0000","last_undegraded":"2024-02-10T07:44:36.340322+0000","last_fullsized":"2024-02-10T07:44:36.340322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:43:38.357448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":28,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342158+0000","last_change":"2024-02-10T07:44:07.511566+0000","last_active":"2024-02-10T07:44:36.342158+0000","last_peered":"2024-02-10T07:44:36.342158+0000","last_clean":"2024-02-10T07:44:36.342158+0000","last_became_active":"2024-02-10T07:44:07.510638+0000","last_became_peered":"2024-02-10T07:44:07.510638+0000","last_unstale":"2024-02-10T07:44:36.342158+0000","last_undegraded":"2024-02-10T07:44:36.342158+0000","last_fullsized":"2024-02-10T07:44:36.342158+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:47:13.935384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"30'13","reported_seq":50,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342211+0000","last_change":"2024-02-10T07:44:09.527972+0000","last_active":"2024-02-10T07:44:36.342211+0000","last_peered":"2024-02-10T07:44:36.342211+0000","last_clean":"2024-02-10T07:44:36.342211+0000","last_became_active":"2024-02-10T07:44:09.527590+0000","last_became_peered":"2024-02-10T07:44:09.527590+0000","last_unstale":"2024-02-10T07:44:36.342211+0000","last_undegraded":"2024-02-10T07:44:36.342211+0000","last_fullsized":"2024-02-10T07:44:36.342211+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:40:43.036140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563316+0000","last_change":"2024-02-10T07:44:11.538135+0000","last_active":"2024-02-10T07:44:42.563316+0000","last_peered":"2024-02-10T07:44:42.563316+0000","last_clean":"2024-02-10T07:44:42.563316+0000","last_became_active":"2024-02-10T07:44:11.537399+0000","last_became_peered":"2024-02-10T07:44:11.537399+0000","last_unstale":"2024-02-10T07:44:42.563316+0000","last_undegraded":"2024-02-10T07:44:42.563316+0000","last_fullsized":"2024-02-10T07:44:42.563316+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:44:50.343670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342069+0000","last_change":"2024-02-10T07:44:13.555904+0000","last_active":"2024-02-10T07:44:36.342069+0000","last_peered":"2024-02-10T07:44:36.342069+0000","last_clean":"2024-02-10T07:44:36.342069+0000","last_became_active":"2024-02-10T07:44:13.555614+0000","last_became_peered":"2024-02-10T07:44:13.555614+0000","last_unstale":"2024-02-10T07:44:36.342069+0000","last_undegraded":"2024-02-10T07:44:36.342069+0000","last_fullsized":"2024-02-10T07:44:36.342069+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:48:54.884304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342099+0000","last_change":"2024-02-10T07:44:07.514113+0000","last_active":"2024-02-10T07:44:36.342099+0000","last_peered":"2024-02-10T07:44:36.342099+0000","last_clean":"2024-02-10T07:44:36.342099+0000","last_became_active":"2024-02-10T07:44:07.513832+0000","last_became_peered":"2024-02-10T07:44:07.513832+0000","last_unstale":"2024-02-10T07:44:36.342099+0000","last_undegraded":"2024-02-10T07:44:36.342099+0000","last_fullsized":"2024-02-10T07:44:36.342099+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:43:27.602809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"30'12","reported_seq":41,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341561+0000","last_change":"2024-02-10T07:44:09.527158+0000","last_active":"2024-02-10T07:44:36.341561+0000","last_peered":"2024-02-10T07:44:36.341561+0000","last_clean":"2024-02-10T07:44:36.341561+0000","last_became_active":"2024-02-10T07:44:09.526342+0000","last_became_peered":"2024-02-10T07:44:09.526342+0000","last_unstale":"2024-02-10T07:44:36.341561+0000","last_undegraded":"2024-02-10T07:44:36.341561+0000","last_fullsized":"2024-02-10T07:44:36.341561+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:38:25.088256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340281+0000","last_change":"2024-02-10T07:44:11.538858+0000","last_active":"2024-02-10T07:44:36.340281+0000","last_peered":"2024-02-10T07:44:36.340281+0000","last_clean":"2024-02-10T07:44:36.340281+0000","last_became_active":"2024-02-10T07:44:11.538620+0000","last_became_peered":"2024-02-10T07:44:11.538620+0000","last_unstale":"2024-02-10T07:44:36.340281+0000","last_undegraded":"2024-02-10T07:44:36.340281+0000","last_fullsized":"2024-02-10T07:44:36.340281+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:24:26.479799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341504+0000","last_change":"2024-02-10T07:44:13.555720+0000","last_active":"2024-02-10T07:44:36.341504+0000","last_peered":"2024-02-10T07:44:36.341504+0000","last_clean":"2024-02-10T07:44:36.341504+0000","last_became_active":"2024-02-10T07:44:13.554995+0000","last_became_peered":"2024-02-10T07:44:13.554995+0000","last_unstale":"2024-02-10T07:44:36.341504+0000","last_undegraded":"2024-02-10T07:44:36.341504+0000","last_fullsized":"2024-02-10T07:44:36.341504+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T10:52:41.761034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"30'8","reported_seq":36,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563632+0000","last_change":"2024-02-10T07:44:11.542900+0000","last_active":"2024-02-10T07:44:42.563632+0000","last_peered":"2024-02-10T07:44:42.563632+0000","last_clean":"2024-02-10T07:44:42.563632+0000","last_became_active":"2024-02-10T07:44:11.541988+0000","last_became_peered":"2024-02-10T07:44:11.541988+0000","last_unstale":"2024-02-10T07:44:42.563632+0000","last_undegraded":"2024-02-10T07:44:42.563632+0000","last_fullsized":"2024-02-10T07:44:42.563632+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:20:46.650085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340300+0000","last_change":"2024-02-10T07:44:07.514719+0000","last_active":"2024-02-10T07:44:36.340300+0000","last_peered":"2024-02-10T07:44:36.340300+0000","last_clean":"2024-02-10T07:44:36.340300+0000","last_became_active":"2024-02-10T07:44:07.513918+0000","last_became_peered":"2024-02-10T07:44:07.513918+0000","last_unstale":"2024-02-10T07:44:36.340300+0000","last_undegraded":"2024-02-10T07:44:36.340300+0000","last_fullsized":"2024-02-10T07:44:36.340300+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:44:30.306094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"30'12","reported_seq":46,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340633+0000","last_change":"2024-02-10T07:44:09.522028+0000","last_active":"2024-02-10T07:44:36.340633+0000","last_peered":"2024-02-10T07:44:36.340633+0000","last_clean":"2024-02-10T07:44:36.340633+0000","last_became_active":"2024-02-10T07:44:09.521610+0000","last_became_peered":"2024-02-10T07:44:09.521610+0000","last_unstale":"2024-02-10T07:44:36.340633+0000","last_undegraded":"2024-02-10T07:44:36.340633+0000","last_fullsized":"2024-02-10T07:44:36.340633+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:26:18.457156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341565+0000","last_change":"2024-02-10T07:44:13.556306+0000","last_active":"2024-02-10T07:44:36.341565+0000","last_peered":"2024-02-10T07:44:36.341565+0000","last_clean":"2024-02-10T07:44:36.341565+0000","last_became_active":"2024-02-10T07:44:13.555458+0000","last_became_peered":"2024-02-10T07:44:13.555458+0000","last_unstale":"2024-02-10T07:44:36.341565+0000","last_undegraded":"2024-02-10T07:44:36.341565+0000","last_fullsized":"2024-02-10T07:44:36.341565+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:17:19.881762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341542+0000","last_change":"2024-02-10T07:44:11.537016+0000","last_active":"2024-02-10T07:44:36.341542+0000","last_peered":"2024-02-10T07:44:36.341542+0000","last_clean":"2024-02-10T07:44:36.341542+0000","last_became_active":"2024-02-10T07:44:11.536487+0000","last_became_peered":"2024-02-10T07:44:11.536487+0000","last_unstale":"2024-02-10T07:44:36.341542+0000","last_undegraded":"2024-02-10T07:44:36.341542+0000","last_fullsized":"2024-02-10T07:44:36.341542+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T11:02:55.450979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339427+0000","last_change":"2024-02-10T07:44:07.509499+0000","last_active":"2024-02-10T07:44:36.339427+0000","last_peered":"2024-02-10T07:44:36.339427+0000","last_clean":"2024-02-10T07:44:36.339427+0000","last_became_active":"2024-02-10T07:44:07.509038+0000","last_became_peered":"2024-02-10T07:44:07.509038+0000","last_unstale":"2024-02-10T07:44:36.339427+0000","last_undegraded":"2024-02-10T07:44:36.339427+0000","last_fullsized":"2024-02-10T07:44:36.339427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T16:55:24.697151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.8","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340703+0000","last_change":"2024-02-10T07:44:09.522567+0000","last_active":"2024-02-10T07:44:36.340703+0000","last_peered":"2024-02-10T07:44:36.340703+0000","last_clean":"2024-02-10T07:44:36.340703+0000","last_became_active":"2024-02-10T07:44:09.522239+0000","last_became_peered":"2024-02-10T07:44:09.522239+0000","last_unstale":"2024-02-10T07:44:36.340703+0000","last_undegraded":"2024-02-10T07:44:36.340703+0000","last_fullsized":"2024-02-10T07:44:36.340703+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:35:29.808886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342861+0000","last_change":"2024-02-10T07:44:13.555899+0000","last_active":"2024-02-10T07:44:36.342861+0000","last_peered":"2024-02-10T07:44:36.342861+0000","last_clean":"2024-02-10T07:44:36.342861+0000","last_became_active":"2024-02-10T07:44:13.555267+0000","last_became_peered":"2024-02-10T07:44:13.555267+0000","last_unstale":"2024-02-10T07:44:36.342861+0000","last_undegraded":"2024-02-10T07:44:36.342861+0000","last_fullsized":"2024-02-10T07:44:36.342861+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:05:55.378474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341105+0000","last_change":"2024-02-10T07:44:07.509885+0000","last_active":"2024-02-10T07:44:36.341105+0000","last_peered":"2024-02-10T07:44:36.341105+0000","last_clean":"2024-02-10T07:44:36.341105+0000","last_became_active":"2024-02-10T07:44:07.509616+0000","last_became_peered":"2024-02-10T07:44:07.509616+0000","last_unstale":"2024-02-10T07:44:36.341105+0000","last_undegraded":"2024-02-10T07:44:36.341105+0000","last_fullsized":"2024-02-10T07:44:36.341105+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:27:36.467289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"30'9","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339931+0000","last_change":"2024-02-10T07:44:09.526656+0000","last_active":"2024-02-10T07:44:36.339931+0000","last_peered":"2024-02-10T07:44:36.339931+0000","last_clean":"2024-02-10T07:44:36.339931+0000","last_became_active":"2024-02-10T07:44:09.526331+0000","last_became_peered":"2024-02-10T07:44:09.526331+0000","last_unstale":"2024-02-10T07:44:36.339931+0000","last_undegraded":"2024-02-10T07:44:36.339931+0000","last_fullsized":"2024-02-10T07:44:36.339931+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:12:44.951278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341972+0000","last_change":"2024-02-10T07:44:11.542972+0000","last_active":"2024-02-10T07:44:36.341972+0000","last_peered":"2024-02-10T07:44:36.341972+0000","last_clean":"2024-02-10T07:44:36.341972+0000","last_became_active":"2024-02-10T07:44:11.542621+0000","last_became_peered":"2024-02-10T07:44:11.542621+0000","last_unstale":"2024-02-10T07:44:36.341972+0000","last_undegraded":"2024-02-10T07:44:36.341972+0000","last_fullsized":"2024-02-10T07:44:36.341972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T15:39:02.475530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340672+0000","last_change":"2024-02-10T07:44:13.555488+0000","last_active":"2024-02-10T07:44:36.340672+0000","last_peered":"2024-02-10T07:44:36.340672+0000","last_clean":"2024-02-10T07:44:36.340672+0000","last_became_active":"2024-02-10T07:44:13.555148+0000","last_became_peered":"2024-02-10T07:44:13.555148+0000","last_unstale":"2024-02-10T07:44:36.340672+0000","last_undegraded":"2024-02-10T07:44:36.340672+0000","last_fullsized":"2024-02-10T07:44:36.340672+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:20:10.111608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"30'8","reported_seq":39,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:42.563623+0000","last_change":"2024-02-10T07:44:11.543200+0000","last_active":"2024-02-10T07:44:42.563623+0000","last_peered":"2024-02-10T07:44:42.563623+0000","last_clean":"2024-02-10T07:44:42.563623+0000","last_became_active":"2024-02-10T07:44:11.541833+0000","last_became_peered":"2024-02-10T07:44:11.541833+0000","last_unstale":"2024-02-10T07:44:42.563623+0000","last_undegraded":"2024-02-10T07:44:42.563623+0000","last_fullsized":"2024-02-10T07:44:42.563623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:41:12.386529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341841+0000","last_change":"2024-02-10T07:44:07.509973+0000","last_active":"2024-02-10T07:44:36.341841+0000","last_peered":"2024-02-10T07:44:36.341841+0000","last_clean":"2024-02-10T07:44:36.341841+0000","last_became_active":"2024-02-10T07:44:07.509704+0000","last_became_peered":"2024-02-10T07:44:07.509704+0000","last_unstale":"2024-02-10T07:44:36.341841+0000","last_undegraded":"2024-02-10T07:44:36.341841+0000","last_fullsized":"2024-02-10T07:44:36.341841+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T14:38:30.139571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"30'19","reported_seq":54,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341176+0000","last_change":"2024-02-10T07:44:09.521999+0000","last_active":"2024-02-10T07:44:36.341176+0000","last_peered":"2024-02-10T07:44:36.341176+0000","last_clean":"2024-02-10T07:44:36.341176+0000","last_became_active":"2024-02-10T07:44:09.521622+0000","last_became_peered":"2024-02-10T07:44:09.521622+0000","last_unstale":"2024-02-10T07:44:36.341176+0000","last_undegraded":"2024-02-10T07:44:36.341176+0000","last_fullsized":"2024-02-10T07:44:36.341176+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T08:03:38.850480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341637+0000","last_change":"2024-02-10T07:44:13.555408+0000","last_active":"2024-02-10T07:44:36.341637+0000","last_peered":"2024-02-10T07:44:36.341637+0000","last_clean":"2024-02-10T07:44:36.341637+0000","last_became_active":"2024-02-10T07:44:13.555112+0000","last_became_peered":"2024-02-10T07:44:13.555112+0000","last_unstale":"2024-02-10T07:44:36.341637+0000","last_undegraded":"2024-02-10T07:44:36.341637+0000","last_fullsized":"2024-02-10T07:44:36.341637+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:13:16.941128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342492+0000","last_change":"2024-02-10T07:44:07.510831+0000","last_active":"2024-02-10T07:44:36.342492+0000","last_peered":"2024-02-10T07:44:36.342492+0000","last_clean":"2024-02-10T07:44:36.342492+0000","last_became_active":"2024-02-10T07:44:07.510539+0000","last_became_peered":"2024-02-10T07:44:07.510539+0000","last_unstale":"2024-02-10T07:44:36.342492+0000","last_undegraded":"2024-02-10T07:44:36.342492+0000","last_fullsized":"2024-02-10T07:44:36.342492+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T12:58:30.733385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"30'17","reported_seq":51,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341727+0000","last_change":"2024-02-10T07:44:09.522588+0000","last_active":"2024-02-10T07:44:36.341727+0000","last_peered":"2024-02-10T07:44:36.341727+0000","last_clean":"2024-02-10T07:44:36.341727+0000","last_became_active":"2024-02-10T07:44:09.522101+0000","last_became_peered":"2024-02-10T07:44:09.522101+0000","last_unstale":"2024-02-10T07:44:36.341727+0000","last_undegraded":"2024-02-10T07:44:36.341727+0000","last_fullsized":"2024-02-10T07:44:36.341727+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:51:40.634580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343272+0000","last_change":"2024-02-10T07:44:11.543761+0000","last_active":"2024-02-10T07:44:36.343272+0000","last_peered":"2024-02-10T07:44:36.343272+0000","last_clean":"2024-02-10T07:44:36.343272+0000","last_became_active":"2024-02-10T07:44:11.542721+0000","last_became_peered":"2024-02-10T07:44:11.542721+0000","last_unstale":"2024-02-10T07:44:36.343272+0000","last_undegraded":"2024-02-10T07:44:36.343272+0000","last_fullsized":"2024-02-10T07:44:36.343272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:24:08.373513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.338869+0000","last_change":"2024-02-10T07:44:13.555283+0000","last_active":"2024-02-10T07:44:36.338869+0000","last_peered":"2024-02-10T07:44:36.338869+0000","last_clean":"2024-02-10T07:44:36.338869+0000","last_became_active":"2024-02-10T07:44:13.554841+0000","last_became_peered":"2024-02-10T07:44:13.554841+0000","last_unstale":"2024-02-10T07:44:36.338869+0000","last_undegraded":"2024-02-10T07:44:36.338869+0000","last_fullsized":"2024-02-10T07:44:36.338869+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T17:33:07.897574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343035+0000","last_change":"2024-02-10T07:44:07.513531+0000","last_active":"2024-02-10T07:44:36.343035+0000","last_peered":"2024-02-10T07:44:36.343035+0000","last_clean":"2024-02-10T07:44:36.343035+0000","last_became_active":"2024-02-10T07:44:07.512810+0000","last_became_peered":"2024-02-10T07:44:07.512810+0000","last_unstale":"2024-02-10T07:44:36.343035+0000","last_undegraded":"2024-02-10T07:44:36.343035+0000","last_fullsized":"2024-02-10T07:44:36.343035+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T09:25:29.034573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"30'10","reported_seq":38,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.342864+0000","last_change":"2024-02-10T07:44:09.523164+0000","last_active":"2024-02-10T07:44:36.342864+0000","last_peered":"2024-02-10T07:44:36.342864+0000","last_clean":"2024-02-10T07:44:36.342864+0000","last_became_active":"2024-02-10T07:44:09.522703+0000","last_became_peered":"2024-02-10T07:44:09.522703+0000","last_unstale":"2024-02-10T07:44:36.342864+0000","last_undegraded":"2024-02-10T07:44:36.342864+0000","last_fullsized":"2024-02-10T07:44:36.342864+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:43:32.088479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.339562+0000","last_change":"2024-02-10T07:44:11.537341+0000","last_active":"2024-02-10T07:44:36.339562+0000","last_peered":"2024-02-10T07:44:36.339562+0000","last_clean":"2024-02-10T07:44:36.339562+0000","last_became_active":"2024-02-10T07:44:11.536481+0000","last_became_peered":"2024-02-10T07:44:11.536481+0000","last_unstale":"2024-02-10T07:44:36.339562+0000","last_undegraded":"2024-02-10T07:44:36.339562+0000","last_fullsized":"2024-02-10T07:44:36.339562+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:55:57.374316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340165+0000","last_change":"2024-02-10T07:44:13.555875+0000","last_active":"2024-02-10T07:44:36.340165+0000","last_peered":"2024-02-10T07:44:36.340165+0000","last_clean":"2024-02-10T07:44:36.340165+0000","last_became_active":"2024-02-10T07:44:13.555353+0000","last_became_peered":"2024-02-10T07:44:13.555353+0000","last_unstale":"2024-02-10T07:44:36.340165+0000","last_undegraded":"2024-02-10T07:44:36.340165+0000","last_fullsized":"2024-02-10T07:44:36.340165+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":28,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:12.534823+0000","last_clean_scrub_stamp":"2024-02-10T07:44:12.534823+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T19:37:22.580056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":27,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.341828+0000","last_change":"2024-02-10T07:44:07.511975+0000","last_active":"2024-02-10T07:44:36.341828+0000","last_peered":"2024-02-10T07:44:36.341828+0000","last_clean":"2024-02-10T07:44:36.341828+0000","last_became_active":"2024-02-10T07:44:07.511694+0000","last_became_peered":"2024-02-10T07:44:07.511694+0000","last_unstale":"2024-02-10T07:44:36.341828+0000","last_undegraded":"2024-02-10T07:44:36.341828+0000","last_fullsized":"2024-02-10T07:44:36.341828+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:06.490414+0000","last_clean_scrub_stamp":"2024-02-10T07:44:06.490414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T13:39:41.454606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"30'15","reported_seq":48,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.343054+0000","last_change":"2024-02-10T07:44:09.523985+0000","last_active":"2024-02-10T07:44:36.343054+0000","last_peered":"2024-02-10T07:44:36.343054+0000","last_clean":"2024-02-10T07:44:36.343054+0000","last_became_active":"2024-02-10T07:44:09.523529+0000","last_became_peered":"2024-02-10T07:44:09.523529+0000","last_unstale":"2024-02-10T07:44:36.343054+0000","last_undegraded":"2024-02-10T07:44:36.343054+0000","last_fullsized":"2024-02-10T07:44:36.343054+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":24,"last_epoch_clean":25,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:08.505027+0000","last_clean_scrub_stamp":"2024-02-10T07:44:08.505027+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T07:53:08.906179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-02-10T07:44:36.340889+0000","last_change":"2024-02-10T07:44:11.538107+0000","last_active":"2024-02-10T07:44:36.340889+0000","last_peered":"2024-02-10T07:44:36.340889+0000","last_clean":"2024-02-10T07:44:36.340889+0000","last_became_active":"2024-02-10T07:44:11.537741+0000","last_became_peered":"2024-02-10T07:44:11.537741+0000","last_unstale":"2024-02-10T07:44:36.340889+0000","last_undegraded":"2024-02-10T07:44:36.340889+0000","last_fullsized":"2024-02-10T07:44:36.340889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-10T07:44:10.518492+0000","last_clean_scrub_stamp":"2024-02-10T07:44:10.518492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-02-11T18:04:20.068785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":3},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":3},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":252,"num_read_kb":342,"num_write":267,"num_write_kb":4504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":208,"ondisk_log_size":208,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1456,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":3}],"osd_stats":[{"osd":2,"up_from":18,"seq":77309411343,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708852,"statfs":{"total":95990841344,"available":95957864448,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738393,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sat Feb 10 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.65900000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":129,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93741056,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93708848,"statfs":{"total":95990841344,"available":95957860352,"internally_reserved":0,"allocated":1052672,"data_stored":700115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sat Feb 10 07:44:30 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.47999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":3702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-10T07:44:58.402 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-10T07:44:58.402 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-10T07:44:58.402 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-10T07:44:58.402 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- ceph health --format=json 2024-02-10T07:44:58.645 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:44:58.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:44:58] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:44:59.063 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.064 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:58 smithi022 ceph-mon[25704]: Deploying daemon grafana.a on smithi027 2024-02-10T07:44:59.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:59.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:58 smithi081 ceph-mon[30233]: Deploying daemon grafana.a on smithi027 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: Regenerating cephadm self-signed grafana TLS certificates 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-10T07:44:59.125 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:44:59.125 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:58 smithi027 ceph-mon[29844]: Deploying daemon grafana.a on smithi027 2024-02-10T07:44:59.816 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:44:59 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:44:59.634Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000144124s 2024-02-10T07:45:00.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:44:59 smithi081 ceph-mon[30233]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:00.123 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:44:59 smithi022 ceph-mon[25704]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:00.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:44:59 smithi027 ceph-mon[29844]: pgmap v14: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:00.609 INFO:teuthology.orchestra.run.smithi022.stdout: 2024-02-10T07:45:00.610 INFO:teuthology.orchestra.run.smithi022.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-10T07:45:00.914 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:00 smithi022 ceph-mon[25704]: from='client.? 172.21.15.22:0/2341287627' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:45:01.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:00 smithi081 ceph-mon[30233]: from='client.? 172.21.15.22:0/2341287627' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:45:01.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:00 smithi027 ceph-mon[29844]: from='client.? 172.21.15.22:0/2341287627' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-10T07:45:01.217 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-10T07:45:01.217 INFO:tasks.cephadm:Setup complete, yielding 2024-02-10T07:45:01.218 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-02-10T07:45:01.227 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi022.front.sepia.ceph.com 2024-02-10T07:45:01.228 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'stat -c '"'"'%u %g'"'"' /var/log/ceph | grep '"'"'167 167'"'"'' 2024-02-10T07:45:01.471 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:01.926 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:01 smithi022 ceph-mon[25704]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:01.926 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:01 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:02.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:01 smithi081 ceph-mon[30233]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:02.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:01 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:01 smithi027 ceph-mon[29844]: pgmap v15: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:02.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:01 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:03.107 INFO:teuthology.orchestra.run.smithi022.stdout:167 167 2024-02-10T07:45:03.770 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch status' 2024-02-10T07:45:04.016 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:04.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:03 smithi081 ceph-mon[30233]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:04.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:03 smithi027 ceph-mon[29844]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:04.136 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:03 smithi022 ceph-mon[25704]: pgmap v16: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:05.829 INFO:teuthology.orchestra.run.smithi022.stdout:Backend: cephadm 2024-02-10T07:45:05.830 INFO:teuthology.orchestra.run.smithi022.stdout:Available: Yes 2024-02-10T07:45:05.830 INFO:teuthology.orchestra.run.smithi022.stdout:Paused: No 2024-02-10T07:45:06.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:05 smithi081 ceph-mon[30233]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:06.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:05 smithi027 ceph-mon[29844]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:06.157 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:05 smithi022 ceph-mon[25704]: pgmap v17: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:06.498 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ps' 2024-02-10T07:45:06.739 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:07.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:06 smithi081 ceph-mon[30233]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:07.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:06 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:07.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:06 smithi022 ceph-mon[25704]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:07.102 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:06 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:07.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:06 smithi027 ceph-mon[29844]: from='client.24404 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:07.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:06 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:07.813 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:45:07 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:45:07.637Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.002454068s 2024-02-10T07:45:08.091 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:07 smithi022 ceph-mon[25704]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:08.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:07 smithi081 ceph-mon[30233]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:08.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:07 smithi027 ceph-mon[29844]: pgmap v18: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:08.591 INFO:teuthology.orchestra.run.smithi022.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-02-10T07:45:08.591 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager.a smithi022 *:9093,9094 starting - - - - 2024-02-10T07:45:08.591 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.x smithi027 *:8443,8765 running (2m) 29s ago 2m 448M - 19.0.0-1361-g290212a1 c328ab9dcfc7 df0bce0f8d4c 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.y smithi022 *:9283,8765 running (4m) 26s ago 4m 477M - 19.0.0-1361-g290212a1 c328ab9dcfc7 cd15e964158b 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:mgr.z smithi081 *:8443,8765 running (2m) 30s ago 2m 447M - 19.0.0-1361-g290212a1 c328ab9dcfc7 c138fc961113 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:mon.a smithi022 running (4m) 26s ago 4m 50.2M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 dabff93c593b 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:mon.b smithi027 running (2m) 29s ago 2m 41.6M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 6e3838a3707c 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:mon.c smithi081 running (2m) 30s ago 2m 47.2M 2048M 19.0.0-1361-g290212a1 c328ab9dcfc7 25a52a7beff0 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:osd.0 smithi022 running (2m) 26s ago 2m 60.8M 15.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 690f17a4facb 2024-02-10T07:45:08.592 INFO:teuthology.orchestra.run.smithi022.stdout:osd.1 smithi027 running (106s) 29s ago 105s 60.4M 15.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 dcb2deb494ef 2024-02-10T07:45:08.593 INFO:teuthology.orchestra.run.smithi022.stdout:osd.2 smithi081 running (81s) 30s ago 81s 59.0M 16.6G 19.0.0-1361-g290212a1 c328ab9dcfc7 7ef97597a88a 2024-02-10T07:45:08.593 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus.a smithi027 *:9095 running (42s) 29s ago 40s 26.7M - 2.43.0 a07b618ecd1d fe7fa7590d3f 2024-02-10T07:45:08.593 INFO:teuthology.orchestra.run.smithi022.stdout:rgw.foo.a smithi022 *:80 running (63s) 26s ago 62s 78.8M - 19.0.0-1361-g290212a1 c328ab9dcfc7 ee3e3e840ba5 2024-02-10T07:45:08.603 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:08 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:45:08] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:09.115 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls' 2024-02-10T07:45:09.355 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:10.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:09 smithi081 ceph-mon[30233]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:10.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:09 smithi081 ceph-mon[30233]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:10.100 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:09 smithi022 ceph-mon[25704]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:10.100 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:09 smithi022 ceph-mon[25704]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:10.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:09 smithi027 ceph-mon[29844]: pgmap v19: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:10.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:09 smithi027 ceph-mon[29844]: from='client.24410 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:11.124 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:10 smithi027 systemd[1]: Starting Ceph grafana.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:alertmanager ?:9093,9094 0/1 - 60s smithi022=a;count:1 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:grafana ?:3000 0/1 - 57s smithi027=a;count:1 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:mgr 3/3 33s ago 2m smithi022=y;smithi027=x;smithi081=z;count:3 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:mon 3/3 33s ago 3m smithi022:172.21.15.22=a;smithi027:172.21.15.27=b;smithi081:172.21.15.81=c;count:3 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:node-exporter ?:9100 0/3 - 63s smithi022=a;smithi027=b;smithi081=c;count:3 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:osd 3 33s ago - 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:prometheus ?:9095 1/1 32s ago 66s smithi027=a;count:1 2024-02-10T07:45:11.178 INFO:teuthology.orchestra.run.smithi022.stdout:rgw.foo.a ?:80 1/1 29s ago 65s smithi022=foo.a;count:1 2024-02-10T07:45:11.422 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 podman[42908]: 2024-02-10 07:45:11.071018245 +0000 UTC m=+0.017322236 image pull a0ea5e3c46d9434bd173d585b7c2638c7451eaf2543d1d22f9e78442615aaffb quay.io/ceph/ceph-grafana:9.4.12 2024-02-10T07:45:11.422 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 podman[42908]: 2024-02-10 07:45:11.194868327 +0000 UTC m=+0.141172306 container create 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, io.k8s.display-name=Red Hat Universal Base Image 8, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, io.openshift.tags=base rhel8, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , build-date=2022-04-29T11:48:11.055897, version=8.5, distribution-scope=public, description=Ceph Grafana Container, io.openshift.expose-services=, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, io.buildah.version=1.32.0, architecture=x86_64) 2024-02-10T07:45:11.422 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 podman[42908]: 2024-02-10 07:45:11.422215441 +0000 UTC m=+0.368519423 container init 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.openshift.expose-services=, vcs-type=git, io.openshift.tags=base rhel8, name=ubi8, architecture=x86_64, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, release=239.1651231664, build-date=2022-04-29T11:48:11.055897, distribution-scope=public, summary=Grafana Container configured for Ceph mgr/dashboard integration, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, maintainer=Paul Cuzner , io.buildah.version=1.32.0) 2024-02-10T07:45:11.672 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 podman[42908]: 2024-02-10 07:45:11.427527646 +0000 UTC m=+0.373831626 container start 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, description=Ceph Grafana Container, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, build-date=2022-04-29T11:48:11.055897, architecture=x86_64, distribution-scope=public, vendor=Red Hat, Inc., name=ubi8, maintainer=Paul Cuzner , release=239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.k8s.display-name=Red Hat Universal Base Image 8, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.openshift.tags=base rhel8, io.openshift.expose-services=) 2024-02-10T07:45:11.672 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 bash[42908]: 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 systemd[1]: Started Ceph grafana.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600258911Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600444456Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600464381Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600472238Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600478817Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600487013Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.60049315Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600499366Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600505664Z level=info msg="Path Home" path=/usr/share/grafana 2024-02-10T07:45:11.673 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600512447Z level=info msg="Path Data" path=/var/lib/grafana 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600518215Z level=info msg="Path Logs" path=/var/log/grafana 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600524115Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600530064Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=settings t=2024-02-10T07:45:11.600536794Z level=info msg="App mode production" 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=sqlstore t=2024-02-10T07:45:11.600581498Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=sqlstore t=2024-02-10T07:45:11.600598978Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.601492119Z level=info msg="Starting DB migrations" 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.601575574Z level=info msg="Executing migration" id="create migration_log table" 2024-02-10T07:45:11.674 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.602124457Z level=info msg="Executing migration" id="create user table" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.602606552Z level=info msg="Executing migration" id="add unique index user.login" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.603095226Z level=info msg="Executing migration" id="add unique index user.email" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.603576581Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.604107094Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.604510246Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.605229277Z level=info msg="Executing migration" id="create user table v2" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.605628069Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.606039421Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.606464357Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.606784168Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.607154814Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.607548118Z level=info msg="Executing migration" id="Update user table charset" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.607834677Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-02-10T07:45:11.675 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.608258075Z level=info msg="Executing migration" id="Add missing user data" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.608595477Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.609026274Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.609432984Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.609843076Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.610986694Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.611438037Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.611902283Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.612720267Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.613164421Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.61401929Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.614962282Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.61569562Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-02-10T07:45:11.676 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.616733125Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.617463975Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.618205937Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.618872156Z level=info msg="Executing migration" id="create temp_user v2" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.619305551Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.620137823Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-02-10T07:45:11.677 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.620570069Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.621046389Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.622081994Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.622817675Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.623693285Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.624066654Z level=info msg="Executing migration" id="create star table" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.624955802Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.625836104Z level=info msg="Executing migration" id="create org table v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.626671965Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.627518068Z level=info msg="Executing migration" id="create org_user table v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.628407717Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.629380315Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.629919973Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.630360149Z level=info msg="Executing migration" id="Update org table charset" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.631269589Z level=info msg="Executing migration" id="Update org_user table charset" 2024-02-10T07:45:11.678 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.631777585Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.632137408Z level=info msg="Executing migration" id="create dashboard table" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.634851121Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.635346465Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.638225631Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.640835187Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.643642078Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.646253882Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.648964216Z level=info msg="Executing migration" id="create dashboard v2" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.651740953Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.654152824Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.65702382Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.659541476Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.662196363Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-02-10T07:45:11.679 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.664884407Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-02-10T07:45:11.680 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.667471427Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-02-10T07:45:11.680 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.670064517Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-02-10T07:45:11.680 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.672570473Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-02-10T07:45:11.680 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.675165408Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-02-10T07:45:11.852 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch host ls' 2024-02-10T07:45:11.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-mon[29844]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:11.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:11.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:11.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:11.922 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.677824176Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.680466073Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.683198915Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.686023935Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.688639724Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.691333688Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.692000279Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.692900246Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-02-10T07:45:11.923 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.693678416Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.694577918Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.695285791Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.696087517Z level=info msg="Executing migration" id="Update dashboard title length" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.697007214Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.697723828Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.698495322Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.699434035Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.700243863Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-10T07:45:11.924 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.70105009Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-10T07:45:11.925 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.701867876Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-02-10T07:45:11.925 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.702776903Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-02-10T07:45:11.925 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.703676085Z level=info msg="Executing migration" id="Add check_sum column" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.704378561Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.705233457Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.705798461Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.706142124Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.707102775Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.707828408Z level=info msg="Executing migration" id="create data_source table" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.70877773Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.709530647Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.710531347Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.711208476Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-02-10T07:45:11.926 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.712156074Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.713259777Z level=info msg="Executing migration" id="create data_source table v2" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.714118756Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.715085489Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.715834912Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.716842666Z level=info msg="Executing migration" id="Add column with_credentials" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.717514762Z level=info msg="Executing migration" id="Add secure json data column" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.718541824Z level=info msg="Executing migration" id="Update data_source table charset" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.719269863Z level=info msg="Executing migration" id="Update initial version to 1" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.72020065Z level=info msg="Executing migration" id="Add read_only data column" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.723006631Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-02-10T07:45:11.927 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.725629475Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.728460704Z level=info msg="Executing migration" id="Add uid column" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.731234289Z level=info msg="Executing migration" id="Update uid value" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.733839549Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.736689929Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.739405602Z level=info msg="Executing migration" id="create api_key table" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.742341903Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.745117375Z level=info msg="Executing migration" id="add index api_key.key" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.747829349Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.750668422Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-02-10T07:45:11.928 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.753269285Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.756042331Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.758846537Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.761671146Z level=info msg="Executing migration" id="create api_key table v2" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.762082932Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.764840511Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.767603074Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.770078712Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.770423085Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.773107658Z level=info msg="Executing migration" id="Update api_key table charset" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.773431759Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-02-10T07:45:11.929 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.775873571Z level=info msg="Executing migration" id="Add service account foreign key" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.778502716Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.77888561Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.781411283Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.784101162Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.786790013Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.787325679Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.789994021Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.79062314Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.792617826Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.795236836Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.795548779Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-02-10T07:45:11.930 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.798080748Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.798658369Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.799237177Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.800029109Z level=info msg="Executing migration" id="create quota table v1" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.800433191Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.801226748Z level=info msg="Executing migration" id="Update quota table charset" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.801987191Z level=info msg="Executing migration" id="create plugin_setting table" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.802972093Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.803433722Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.804341659Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.804986818Z level=info msg="Executing migration" id="create session table" 2024-02-10T07:45:11.931 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.805783256Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.806126082Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.806967931Z level=info msg="Executing migration" id="create playlist table v2" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.807406894Z level=info msg="Executing migration" id="create playlist item table v2" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.807792068Z level=info msg="Executing migration" id="Update playlist table charset" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.808681429Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.808993976Z level=info msg="Executing migration" id="drop preferences table v2" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.809977097Z level=info msg="Executing migration" id="drop preferences table v3" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.810697904Z level=info msg="Executing migration" id="create preferences table v3" 2024-02-10T07:45:11.932 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.811695608Z level=info msg="Executing migration" id="Update preferences table charset" 2024-02-10T07:45:11.933 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.812013831Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-02-10T07:45:11.933 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.812905103Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-02-10T07:45:11.933 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.813701594Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-02-10T07:45:11.933 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.814571981Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-02-10T07:45:11.933 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.815179833Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.816028619Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.816996132Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.8177297Z level=info msg="Executing migration" id="create alert table v1" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.818225832Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.819205608Z level=info msg="Executing migration" id="add index alert state" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.81990163Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.820890893Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.821444583Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.822348774Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.823303287Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.826029132Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.826443727Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-10T07:45:11.934 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.829093347Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.831941252Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.834461865Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.834880033Z level=info msg="Executing migration" id="Add column is_default" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.835726081Z level=info msg="Executing migration" id="Add column frequency" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.836372067Z level=info msg="Executing migration" id="Add column send_reminder" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.837145468Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.837787753Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.838229037Z level=info msg="Executing migration" id="Update alert table charset" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.838506753Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.8387869Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.839220594Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-10T07:45:11.935 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.839650107Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.840057568Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.842947922Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.845773242Z level=info msg="Executing migration" id="Add for to alert table" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.848722746Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.851657885Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.8542716Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.854771297Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.85751882Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.86040899Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.86313528Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.865925282Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-02-10T07:45:11.936 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.868683793Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.869022744Z level=info msg="Executing migration" id="create annotation table v5" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.871843699Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.874763415Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.877606482Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.878226121Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.881089113Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.883918982Z level=info msg="Executing migration" id="Update annotation table charset" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.886574027Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.887270176Z level=info msg="Executing migration" id="Drop category_id index" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.889947474Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.892644469Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.895394572Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-10T07:45:11.937 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.895954407Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.898576901Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.901460887Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.9039667Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.904408824Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.907109482Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.907501248Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.909633367Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-02-10T07:45:11.939 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.910364519Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.912490599Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.912959314Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.915617859Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.916310732Z level=info msg="Executing migration" id="Add epoch_end column" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.917192333Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.917635525Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.918640875Z level=info msg="Executing migration" id="Move region to single row" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.919351751Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.920231459Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.92090045Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.92171531Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-10T07:45:11.940 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.922680089Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.923359879Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.924185143Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.924528027Z level=info msg="Executing migration" id="create test_data table" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.924952029Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.925376031Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.926192783Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.927037605Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.927378037Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.928194202Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.928521667Z level=info msg="Executing migration" id="create team table" 2024-02-10T07:45:11.941 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.929078039Z level=info msg="Executing migration" id="add index team.org_id" 2024-02-10T07:45:12.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:11 smithi081 ceph-mon[30233]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:12.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:11 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:11 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:11 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.098 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:11 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.098 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.929963202Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.930425247Z level=info msg="Executing migration" id="create team member table" 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.931306634Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.932158913Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.933029297Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-02-10T07:45:12.173 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.933487652Z level=info msg="Executing migration" id="Add column email to team table" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.934372573Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.935217163Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.936069208Z level=info msg="Executing migration" id="create dashboard acl table" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.936509208Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.937370469Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.938165847Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.938640147Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.939135596Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.939938516Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.940413581Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.940882895Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.941337218Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-02-10T07:45:12.174 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.941629491Z level=info msg="Executing migration" id="create tag table" 2024-02-10T07:45:12.175 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.942038966Z level=info msg="Executing migration" id="add index tag.key_value" 2024-02-10T07:45:12.175 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.942478064Z level=info msg="Executing migration" id="create login attempt table" 2024-02-10T07:45:12.175 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.942880173Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.943321419Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.94371436Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.945601308Z level=info msg="Executing migration" id="create login_attempt v2" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.946015637Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.946449575Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.946745839Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.947128176Z level=info msg="Executing migration" id="create user auth table" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.94751672Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.947954029Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.948265988Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.949012732Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-02-10T07:45:12.176 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.949751838Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.950499444Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.951280688Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.9517162Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.95247468Z level=info msg="Executing migration" id="create server_lock table" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.952911257Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.953370767Z level=info msg="Executing migration" id="create user auth token table" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.953800968Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.954621074Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.955073722Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.955501604Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.956283055Z level=info msg="Executing migration" id="create cache_data table" 2024-02-10T07:45:12.177 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.956704297Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.957173843Z level=info msg="Executing migration" id="create short_url table v1" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.957603474Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.958436464Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.958737707Z level=info msg="Executing migration" id="delete alert_definition table" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.959064346Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.959509733Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.962387355Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.965154428Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.967951991Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.968463195Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.971356478Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.97199523Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-02-10T07:45:12.178 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.974014026Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.976822968Z level=info msg="Executing migration" id="drop alert_definition table" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.979659441Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.982473699Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.985066786Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.98796364Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.990695923Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.991028293Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.99192987Z level=info msg="Executing migration" id="create alert_instance table" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.992797065Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.993629911Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.996718229Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:11 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:11.999294632Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-10T07:45:12.179 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.001906467Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-02-10T07:45:12.180 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.004628558Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-02-10T07:45:12.180 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.008207742Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.011609849Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.014281768Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.017003112Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.019682986Z level=info msg="Executing migration" id="create alert_rule table" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.022229294Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.025116524Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.027842376Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.030573494Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.033210258Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.03602609Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.038791363Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-02-10T07:45:12.181 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.041582465Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.044049943Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.046780586Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.049660224Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.052160749Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.054673781Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.057298649Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.059879416Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.062692774Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.065225021Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.067665769Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.070353268Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.072987922Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-02-10T07:45:12.182 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.075683197Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.078248171Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.080954606Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.083676579Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.086448088Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.089031848Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.089703443Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.090567129Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.091386968Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.092303129Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.093049105Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.093925407Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-02-10T07:45:12.183 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.094700112Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.095521896Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.096410813Z level=info msg="Executing migration" id="create provenance_type table" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.097224384Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.09816719Z level=info msg="Executing migration" id="create alert_image table" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.09885944Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.099644038Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.100492843Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.101473179Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.102165396Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.102244986Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.102929564Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.103895176Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-02-10T07:45:12.184 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.104618016Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.104733545Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.104862691Z level=info msg="alerts found to migrate" alerts=0 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.105497196Z level=info msg="Executing migration" id="create library_element table v1" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.106280253Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.107240728Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.107950221Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.108938709Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.10968734Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.110664557Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.111403905Z level=info msg="Executing migration" id="create data_keys table" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.112272605Z level=info msg="Executing migration" id="create secrets table" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.113044211Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-02-10T07:45:12.185 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.117422957Z level=info msg="Executing migration" id="add name column into data_keys" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.118427233Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.119394659Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.123862195Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.128199046Z level=info msg="Executing migration" id="create kv_store table v1" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.129061573Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.131993374Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.134760675Z level=info msg="Executing migration" id="create permission table" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.137443409Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.140165872Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.142926592Z level=info msg="Executing migration" id="create role table" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.145585587Z level=info msg="Executing migration" id="add column display_name" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.148171874Z level=info msg="Executing migration" id="add column group_name" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.150915609Z level=info msg="Executing migration" id="add index role.org_id" 2024-02-10T07:45:12.186 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.153644889Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.156380438Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.159182496Z level=info msg="Executing migration" id="create team role table" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.16177436Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.164393978Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.167173867Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.170074352Z level=info msg="Executing migration" id="create user role table" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.172854581Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.175259764Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-02-10T07:45:12.187 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.177907459Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-02-10T07:45:12.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:11 smithi022 ceph-mon[25704]: pgmap v20: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:12.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:11 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:11 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:11 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:11 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:12.624 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.180605542Z level=info msg="Executing migration" id="create builtin role table" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.183269728Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.185921171Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.188794273Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.191265527Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.19397766Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.196592525Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.199122869Z level=info msg="Executing migration" id="add unique index role.uid" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.201901361Z level=info msg="Executing migration" id="create seed assignment table" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.204640954Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.207131457Z level=info msg="Executing migration" id="add column hidden to role table" 2024-02-10T07:45:12.625 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.209611411Z level=info msg="Executing migration" id="create query_history table v1" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.212435987Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.214977704Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.217697812Z level=info msg="Executing migration" id="teams permissions migration" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.220344046Z level=info msg="Executing migration" id="dashboard permissions" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.222906902Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.223736584Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-02-10T07:45:12.626 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.22457841Z level=info msg="Executing migration" id="alerting notification permissions" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.225537765Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.22616475Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.227062124Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.228144591Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.229013074Z level=info msg="Executing migration" id="create correlation table v1" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.229927841Z level=info msg="Executing migration" id="add index correlations.uid" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.230528813Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.231433021Z level=info msg="Executing migration" id="add correlation config column" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.232614323Z level=info msg="Executing migration" id="create entity_events table" 2024-02-10T07:45:12.627 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.233407819Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.234189523Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.234278547Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.2351208Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.23533159Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.235827065Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.236813639Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.237474546Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.238327126Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.239157947Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.241442556Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T07:45:12.628 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.244078866Z level=info msg="Executing migration" id="Drop public config table" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.246927174Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.249509954Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.252147519Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.254540146Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.257244332Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.260272107Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.262995651Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.265512409Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.268215959Z level=info msg="Executing migration" id="add share column" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.271029729Z level=info msg="Executing migration" id="create default alerting folders" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.273710837Z level=info msg="Executing migration" id="create file table" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.27643786Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-02-10T07:45:12.629 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.278948391Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.279428623Z level=info msg="Executing migration" id="create file_meta table" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.279876832Z level=info msg="Executing migration" id="file table idx: path key" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.280723373Z level=info msg="Executing migration" id="set path collation in file table" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.281544771Z level=info msg="Executing migration" id="managed permissions migration" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.28254398Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.283275772Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.284117302Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.285177317Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.286152478Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.286893669Z level=info msg="Executing migration" id="update group index for alert rules" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.28771656Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.28866528Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.289438989Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-02-10T07:45:12.630 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.290499014Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.291552726Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.292408021Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.30246999Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.303330619Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.30419241Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.307992354Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.308989954Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.309736301Z level=info msg="Executing migration" id="create folder table" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.310731085Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-02-10T07:45:12.631 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.311466572Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.312508578Z level=info msg="Executing migration" id="Update folder title length" 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.313190318Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=migrator t=2024-02-10T07:45:12.314104869Z level=info msg="migrations completed" performed=485 skipped=0 duration=712.553864ms 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=sqlstore t=2024-02-10T07:45:12.314559447Z level=info msg="Created default organization" 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=plugin.loader t=2024-02-10T07:45:12.37431995Z level=info msg="Plugin registered" pluginID=input 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=plugin.loader t=2024-02-10T07:45:12.403913503Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=plugin.loader t=2024-02-10T07:45:12.403933188Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=secrets t=2024-02-10T07:45:12.40396719Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=query_data t=2024-02-10T07:45:12.406120269Z level=info msg="Query Service initialization" 2024-02-10T07:45:12.632 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=live.push_http t=2024-02-10T07:45:12.408453373Z level=info msg="Live Push Gateway initialization" 2024-02-10T07:45:13.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-mon[29844]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:13.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:12 smithi027 ceph-mon[29844]: Deploying daemon node-exporter.a on smithi022 2024-02-10T07:45:13.124 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=infra.usagestats.collector t=2024-02-10T07:45:13.054215866Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=provisioning.datasources t=2024-02-10T07:45:13.055193375Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=provisioning.datasources t=2024-02-10T07:45:13.066334561Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=provisioning.alerting t=2024-02-10T07:45:13.077525067Z level=info msg="starting to provision alerting" 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=provisioning.alerting t=2024-02-10T07:45:13.077545078Z level=info msg="finished to provision alerting" 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=grafanaStorageLogger t=2024-02-10T07:45:13.077656141Z level=info msg="storage starting" 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=ngalert.state.manager t=2024-02-10T07:45:13.077719022Z level=info msg="Warming state cache for startup" 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=ngalert.state.manager t=2024-02-10T07:45:13.077971961Z level=info msg="State cache has been initialized" states=0 duration=251.532µs 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=ticker t=2024-02-10T07:45:13.078072828Z level=info msg=starting first_tick=2024-02-10T07:45:20Z 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=ngalert.multiorg.alertmanager t=2024-02-10T07:45:13.078680297Z level=info msg="starting MultiOrg Alertmanager" 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=http.server t=2024-02-10T07:45:13.080362476Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-02-10T07:45:13.125 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a[42922]: logger=sqlstore.transactions t=2024-02-10T07:45:13.094454736Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-02-10T07:45:13.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:12 smithi022 ceph-mon[25704]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:13.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:12 smithi022 ceph-mon[25704]: Deploying daemon node-exporter.a on smithi022 2024-02-10T07:45:13.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:12 smithi081 ceph-mon[30233]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:13.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:12 smithi081 ceph-mon[30233]: Deploying daemon node-exporter.a on smithi022 2024-02-10T07:45:13.934 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:13 smithi022 ceph-mon[25704]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:14.048 INFO:teuthology.orchestra.run.smithi022.stdout:HOST ADDR LABELS STATUS 2024-02-10T07:45:14.048 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 172.21.15.22 2024-02-10T07:45:14.048 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 172.21.15.27 2024-02-10T07:45:14.048 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 172.21.15.81 2024-02-10T07:45:14.048 INFO:teuthology.orchestra.run.smithi022.stdout:3 hosts in cluster 2024-02-10T07:45:14.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:13 smithi027 ceph-mon[29844]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:14.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:13 smithi081 ceph-mon[30233]: pgmap v21: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:14.952 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch device ls' 2024-02-10T07:45:15.109 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:14 smithi081 ceph-mon[30233]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:15.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:14 smithi027 ceph-mon[29844]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:15.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:14 smithi022 ceph-mon[25704]: from='client.14574 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:15.841 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:16.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:15 smithi027 ceph-mon[29844]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:16.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:15 smithi022 ceph-mon[25704]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:16.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:15 smithi081 ceph-mon[30233]: pgmap v22: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:16.896 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:16 smithi022 podman[59353]: 2024-02-10 07:45:16.896095774 +0000 UTC m=+3.064427697 container create d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:45:17.149 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 podman[59353]: 2024-02-10 07:45:17.13980353 +0000 UTC m=+3.308135463 container init d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:45:17.149 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 podman[59353]: 2024-02-10 07:45:17.145468786 +0000 UTC m=+3.313800708 container start d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:45:17.430 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.149Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T07:45:17.430 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.149Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T07:45:17.431 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.150Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T07:45:17.432 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T07:45:17.433 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T07:45:17.434 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a[59539]: ts=2024-02-10T07:45:17.151Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 bash[59353]: d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c 2024-02-10T07:45:17.435 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:45:17 smithi022 systemd[1]: Started Ceph node-exporter.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:17.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-mon[25704]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:17.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:17 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.699 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:17 smithi027 ceph-mon[29844]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:17.699 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:17 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.699 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:17 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.699 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:17 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.699 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:17 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:17 smithi081 ceph-mon[30233]: pgmap v23: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:17.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:17 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:17 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:17 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:17.848 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:17 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:18.452 INFO:teuthology.orchestra.run.smithi022.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-02-10T07:45:18.452 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB747402EC480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-02-10T07:45:18.452 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme1n1 ssd Linux_195905c835d344ec2706 89.4G Yes 2m ago 2024-02-10T07:45:18.452 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme2n1 ssd Linux_8598b221f58a6e76262d 89.4G Yes 2m ago 2024-02-10T07:45:18.452 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme3n1 ssd Linux_f7bab0afee587262db69 89.4G Yes 2m ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi022 /dev/nvme4n1 ssd Linux_4c6f184022173ed3ece2 89.4G No 2m ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331001C400BGN 372G No 107s ago Has a FileSystem, LVM detected 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 /dev/nvme1n1 ssd Linux_d6b182bfade4c62e29c5 89.4G Yes 107s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 /dev/nvme2n1 ssd Linux_c214e663dd948f2ad906 89.4G Yes 107s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 /dev/nvme3n1 ssd Linux_d7d2419c00bb0d29e2d9 89.4G Yes 107s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi027 /dev/nvme4n1 ssd Linux_2cf39b667b9184791ebb 89.4G No 107s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400N7400BGN 372G No 83s ago Has a FileSystem, LVM detected 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 /dev/nvme1n1 ssd Linux_a3a5c538dee4d86a70f3 89.4G Yes 83s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 /dev/nvme2n1 ssd Linux_d073ecffb9d6e1fc926d 89.4G Yes 83s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 /dev/nvme3n1 ssd Linux_58a182ce31680661db17 89.4G Yes 83s ago 2024-02-10T07:45:18.453 INFO:teuthology.orchestra.run.smithi022.stdout:smithi081 /dev/nvme4n1 ssd Linux_ff3093766c18c2b1ff54 89.4G No 83s ago Has a FileSystem, Insufficient space (<10 extents) on vgs, LVM detected 2024-02-10T07:45:18.624 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:18 smithi027 ceph-mon[29844]: Deploying daemon node-exporter.b on smithi027 2024-02-10T07:45:18.625 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:18 smithi027 bash[43189]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T07:45:18.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:18 smithi022 ceph-mon[25704]: Deploying daemon node-exporter.b on smithi027 2024-02-10T07:45:18.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:18 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:45:18] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:18.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:18 smithi081 ceph-mon[30233]: Deploying daemon node-exporter.b on smithi027 2024-02-10T07:45:19.102 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls --format yaml' 2024-02-10T07:45:19.348 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:19.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:18 smithi027 bash[43189]: Getting image source signatures 2024-02-10T07:45:19.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:18 smithi027 bash[43189]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T07:45:19.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:18 smithi027 bash[43189]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T07:45:19.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:18 smithi027 bash[43189]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T07:45:19.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:19 smithi022 ceph-mon[25704]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:19.694 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:19 smithi022 ceph-mon[25704]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:19.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:19 smithi081 ceph-mon[30233]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:19.847 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:19 smithi081 ceph-mon[30233]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:19.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:19 smithi027 ceph-mon[29844]: pgmap v24: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:19.874 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:19 smithi027 ceph-mon[29844]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:20.577 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 bash[43189]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T07:45:20.578 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 bash[43189]: Writing manifest to image destination 2024-02-10T07:45:20.861 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 podman[43189]: 2024-02-10 07:45:20.545164861 +0000 UTC m=+2.090778567 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T07:45:20.862 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 podman[43189]: 2024-02-10 07:45:20.643577536 +0000 UTC m=+2.189191228 container create ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:45:21.124 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 podman[43189]: 2024-02-10 07:45:20.862293681 +0000 UTC m=+2.407907376 container init ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 podman[43189]: 2024-02-10 07:45:20.867758212 +0000 UTC m=+2.413371907 container start ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.872Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.872Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T07:45:21.125 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T07:45:21.126 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T07:45:21.127 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T07:45:21.128 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.873Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.874Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b[43239]: ts=2024-02-10T07:45:20.874Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 bash[43189]: ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 2024-02-10T07:45:21.129 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:45:20 smithi027 systemd[1]: Started Ceph node-exporter.b for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: alertmanager 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: alertmanager 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout: count: 1 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.189 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi022=a 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:44:10.569158Z' 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: ports: 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: - 9093 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: - 9094 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: running: 0 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: size: 1 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:events: 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:- 2024-02-10T07:44:57.762648Z service:alertmanager [INFO] "service was created" 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: grafana 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: grafana 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: count: 1 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.190 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi027=a 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:spec: 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: anonymous_access: true 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: protocol: https 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:44:13.334937Z' 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: ports: 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: - 3000 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: running: 0 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout: size: 1 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:events: 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:- 2024-02-10T07:45:11.627765Z service:grafana [INFO] "service was created" 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: mgr 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: mgr 2024-02-10T07:45:21.191 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: count: 3 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi022=y 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi027=x 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi081=z 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:42:33.862211Z' 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: last_refresh: '2024-02-10T07:44:38.091701Z' 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: running: 3 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout: size: 3 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.192 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: mon 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: mon 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: count: 3 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi022:172.21.15.22=a 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi027:172.21.15.27=b 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi081:172.21.15.81=c 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:42:05.645592Z' 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: last_refresh: '2024-02-10T07:44:38.091596Z' 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: running: 3 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout: size: 3 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.193 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: node-exporter 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: node-exporter 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: count: 3 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi022=a 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi027=b 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi081=c 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:44:07.642184Z' 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: ports: 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: - 9100 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: running: 0 2024-02-10T07:45:21.194 INFO:teuthology.orchestra.run.smithi022.stdout: size: 3 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:events: 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:- 2024-02-10T07:45:21.031675Z service:node-exporter [INFO] "service was created" 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: osd 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: osd 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:unmanaged: true 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:spec: 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: filter_logic: AND 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: objectstore: bluestore 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: container_image_id: c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: container_image_name: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: last_refresh: '2024-02-10T07:44:38.091784Z' 2024-02-10T07:45:21.195 INFO:teuthology.orchestra.run.smithi022.stdout: running: 3 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: size: 3 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: prometheus 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: prometheus 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: count: 1 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi027=a 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:44:04.990238Z' 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: last_refresh: '2024-02-10T07:44:38.796171Z' 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: ports: 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: - 9095 2024-02-10T07:45:21.196 INFO:teuthology.orchestra.run.smithi022.stdout: running: 1 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: size: 1 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:--- 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:service_type: rgw 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:service_id: foo.a 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:service_name: rgw.foo.a 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:placement: 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: count: 1 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: hosts: 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: - smithi022=foo.a 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout:status: 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: created: '2024-02-10T07:44:05.990339Z' 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: last_refresh: '2024-02-10T07:44:41.904210Z' 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: ports: 2024-02-10T07:45:21.197 INFO:teuthology.orchestra.run.smithi022.stdout: - 80 2024-02-10T07:45:21.198 INFO:teuthology.orchestra.run.smithi022.stdout: running: 1 2024-02-10T07:45:21.198 INFO:teuthology.orchestra.run.smithi022.stdout: size: 1 2024-02-10T07:45:21.764 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 -- bash -c 'ceph orch ls | grep '"'"'^osd '"'"'' 2024-02-10T07:45:22.005 INFO:teuthology.orchestra.run.smithi022.stderr:Inferring config /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/mon.a/config 2024-02-10T07:45:22.267 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:22.268 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.268 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.268 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.268 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: Deploying daemon node-exporter.c on smithi081 2024-02-10T07:45:22.268 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:22 smithi081 ceph-mon[30233]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:22.268 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 systemd[1]: Starting Ceph node-exporter.c for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: Deploying daemon node-exporter.c on smithi081 2024-02-10T07:45:22.276 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:22 smithi022 ceph-mon[25704]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: pgmap v25: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: Deploying daemon node-exporter.c on smithi081 2024-02-10T07:45:22.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:22 smithi027 ceph-mon[29844]: from='mgr.14451 172.21.15.22:0/2719004384' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-02-10T07:45:22.597 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 bash[43358]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-02-10T07:45:23.097 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:23 smithi081 ceph-mon[30233]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T07:45:23.097 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 bash[43358]: Getting image source signatures 2024-02-10T07:45:23.098 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 bash[43358]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-02-10T07:45:23.098 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 bash[43358]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-02-10T07:45:23.098 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:22 smithi081 bash[43358]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-02-10T07:45:23.194 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:23 smithi022 ceph-mon[25704]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T07:45:23.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:23 smithi027 ceph-mon[29844]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""], "format": "yaml"}]: dispatch 2024-02-10T07:45:23.997 INFO:teuthology.orchestra.run.smithi022.stdout:osd 3 45s ago - 2024-02-10T07:45:24.277 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:24 smithi022 ceph-mon[25704]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:24.292 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-mon[30233]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:24.292 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 bash[43358]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-02-10T07:45:24.292 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 bash[43358]: Writing manifest to image destination 2024-02-10T07:45:24.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:24 smithi027 ceph-mon[29844]: pgmap v26: 129 pgs: 129 active+clean; 582 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-02-10T07:45:24.564 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-10T07:45:24.573 INFO:tasks.cephadm:Teardown begin 2024-02-10T07:45:24.574 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:45:24.603 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:45:24.644 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:45:24.678 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-10T07:45:24.679 DEBUG:teuthology.orchestra.run.smithi022:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:45:24.694 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:45:24.711 DEBUG:teuthology.orchestra.run.smithi081:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-10T07:45:24.734 INFO:tasks.cephadm:Stopping all daemons... 2024-02-10T07:45:24.735 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-10T07:45:24.735 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a 2024-02-10T07:45:24.847 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 podman[43358]: 2024-02-10 07:45:24.573072164 +0000 UTC m=+2.400107095 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-02-10T07:45:24.847 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 podman[43358]: 2024-02-10 07:45:24.663038101 +0000 UTC m=+2.490073023 container create 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:45:24.944 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:24 smithi022 systemd[1]: Stopping Ceph mon.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:25.347 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:25 smithi081 ceph-mon[30233]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:25.348 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 podman[43358]: 2024-02-10 07:45:24.934721142 +0000 UTC m=+2.761756064 container init 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:45:25.348 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 podman[43358]: 2024-02-10 07:45:24.940434545 +0000 UTC m=+2.767469468 container start 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:45:25.348 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.945Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-02-10T07:45:25.348 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.945Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-02-10T07:45:25.348 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.945Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.945Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=arp 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=bcache 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=bonding 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=btrfs 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=conntrack 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=cpu 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=diskstats 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=dmi 2024-02-10T07:45:25.349 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=edac 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=entropy 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=filefd 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=filesystem 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=hwmon 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=infiniband 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=ipvs 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=loadavg 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=mdadm 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=meminfo 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=netclass 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=netdev 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=netstat 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=nfs 2024-02-10T07:45:25.350 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=nfsd 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=nvme 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=os 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=pressure 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=rapl 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=schedstat 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=selinux 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=sockstat 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=softnet 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=stat 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=tapestats 2024-02-10T07:45:25.351 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=textfile 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=time 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=uname 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=vmstat 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=xfs 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.946Z caller=node_exporter.go:117 level=info collector=zfs 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.947Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:24 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c[43441]: ts=2024-02-10T07:45:24.947Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:25 smithi081 bash[43358]: 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 2024-02-10T07:45:25.352 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:45:25 smithi081 systemd[1]: Started Ceph node-exporter.c for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:25.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:25 smithi027 ceph-mon[29844]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-mon[25704]: from='client.14592 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-02-10T07:45:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a[25681]: 2024-02-10T07:45:25.067+0000 7f9d54d51700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a[25681]: 2024-02-10T07:45:25.067+0000 7f9d54d51700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-10T07:45:25.444 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 podman[60163]: 2024-02-10 07:45:25.221148537 +0000 UTC m=+0.295373204 container died dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE Bus STOPPING 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE Bus STOPPED 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE Bus STARTING 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE Serving on http://:::9283 2024-02-10T07:45:25.445 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:25 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: [10/Feb/2024:07:45:25] ENGINE Bus STARTED 2024-02-10T07:45:25.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 podman[60163]: 2024-02-10 07:45:25.581162571 +0000 UTC m=+0.655387256 container cleanup dabff93c593ba7617e5bc71cd362d5b5f1c02a1b3f3959585da71b91434ec9d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=main-290212a, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-02-10T07:45:25.865 INFO:journalctl@ceph.mon.a.smithi022.stdout:Feb 10 07:45:25 smithi022 bash[60163]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-a 2024-02-10T07:45:26.003 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.a.service' 2024-02-10T07:45:26.053 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:26.053 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-10T07:45:26.053 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-02-10T07:45:26.053 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.b 2024-02-10T07:45:26.374 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 systemd[1]: Stopping Ceph mon.b for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:26.859 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b[29821]: 2024-02-10T07:45:26.378+0000 7f50b59bb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:26.859 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b[29821]: 2024-02-10T07:45:26.378+0000 7f50b59bb700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-02-10T07:45:26.859 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 podman[43351]: 2024-02-10 07:45:26.624368435 +0000 UTC m=+0.370419815 container died 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=main-290212a) 2024-02-10T07:45:27.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 podman[43351]: 2024-02-10 07:45:26.950017739 +0000 UTC m=+0.696069119 container cleanup 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:45:27.124 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:26 smithi027 bash[43351]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b 2024-02-10T07:45:27.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.b.service' 2024-02-10T07:45:27.414 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:27 smithi027 podman[43371]: 2024-02-10 07:45:27.161485798 +0000 UTC m=+0.535160991 container remove 6e3838a3707c143615466dd554b3721225ae5baba7b066070ac28f120e73fbe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-b, org.label-schema.vendor=CentOS, RELEASE=main-290212a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux ) 2024-02-10T07:45:27.414 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:27 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.b.service: Deactivated successfully. 2024-02-10T07:45:27.414 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:27 smithi027 systemd[1]: Stopped Ceph mon.b for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:27.414 INFO:journalctl@ceph.mon.b.smithi027.stdout:Feb 10 07:45:27 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.b.service: Consumed 3.976s CPU time. 2024-02-10T07:45:27.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:27.437 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-02-10T07:45:27.437 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-02-10T07:45:27.437 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.c 2024-02-10T07:45:27.724 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:27 smithi081 systemd[1]: Stopping Ceph mon.c for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:27.990 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:27 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c[30210]: 2024-02-10T07:45:27.724+0000 7feee2662700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:27.990 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:27 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c[30210]: 2024-02-10T07:45:27.724+0000 7feee2662700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-02-10T07:45:28.265 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:27 smithi081 podman[43518]: 2024-02-10 07:45:27.990880333 +0000 UTC m=+0.364977540 container died 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:45:28.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:28 smithi081 podman[43518]: 2024-02-10 07:45:28.439260934 +0000 UTC m=+0.813358141 container cleanup 25a52a7beff07a812ccd2fe60b2996ae7ccc1fbf0e4e1cf1a55ab2ec704827da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=main-290212a, ceph=True) 2024-02-10T07:45:28.597 INFO:journalctl@ceph.mon.c.smithi081.stdout:Feb 10 07:45:28 smithi081 bash[43518]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mon-c 2024-02-10T07:45:28.694 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:28 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: ::ffff:172.21.15.27 - - [10/Feb/2024:07:45:28] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-02-10T07:45:28.857 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mon.c.service' 2024-02-10T07:45:28.905 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:28.906 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-02-10T07:45:28.906 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-10T07:45:28.906 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y 2024-02-10T07:45:29.194 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:28 smithi022 systemd[1]: Stopping Ceph mgr.y for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:29.493 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:45:29.220+0000 7f4360bf9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:29.494 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y[25961]: 2024-02-10T07:45:29.220+0000 7f4360bf9700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-10T07:45:29.494 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 podman[60305]: 2024-02-10 07:45:29.252249102 +0000 UTC m=+0.151532873 container died cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=main-290212a, org.label-schema.license=GPLv2) 2024-02-10T07:45:29.794 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 podman[60305]: 2024-02-10 07:45:29.62617667 +0000 UTC m=+0.525460440 container cleanup cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, maintainer=Guillaume Abrioux , RELEASE=main-290212a, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-02-10T07:45:29.794 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 bash[60305]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y 2024-02-10T07:45:30.044 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service' 2024-02-10T07:45:30.067 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:29 smithi022 podman[60324]: 2024-02-10 07:45:29.794335904 +0000 UTC m=+0.542534510 container remove cd15e964158befb479b06a6c3e8ab33e088986ed3aa2fe2b9c10312c42af4d76 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:290212a1bfd33563ba76ee5b7f0b944fdb6c5d90, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-y, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=main-290212a, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-02-10T07:45:30.067 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:30 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service: Deactivated successfully. 2024-02-10T07:45:30.067 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:30 smithi022 systemd[1]: Stopped Ceph mgr.y for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:30.067 INFO:journalctl@ceph.mgr.y.smithi022.stdout:Feb 10 07:45:30 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.y.service: Consumed 54.947s CPU time. 2024-02-10T07:45:30.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:30.079 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-10T07:45:30.079 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-10T07:45:30.080 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x 2024-02-10T07:45:30.374 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 systemd[1]: Stopping Ceph mgr.x for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:30.704 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 podman[43484]: 2024-02-10 07:45:30.444738455 +0000 UTC m=+0.166128258 container died df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-02-10T07:45:30.971 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 podman[43484]: 2024-02-10 07:45:30.779137131 +0000 UTC m=+0.500526933 container cleanup df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True) 2024-02-10T07:45:30.972 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 bash[43484]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x 2024-02-10T07:45:31.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x.service' 2024-02-10T07:45:31.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 podman[43503]: 2024-02-10 07:45:30.972026419 +0000 UTC m=+0.523084612 container remove df0bce0f8d4cfb8eca1c3367a9355741cde7a5a0d6dcf793aba0273f57bf58c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.build-date=20240131, ceph=True) 2024-02-10T07:45:31.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:30 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:45:31.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:31 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x.service: Failed with result 'exit-code'. 2024-02-10T07:45:31.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:31 smithi027 systemd[1]: Stopped Ceph mgr.x for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:45:31.239 INFO:journalctl@ceph.mgr.x.smithi027.stdout:Feb 10 07:45:31 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.x.service: Consumed 16.988s CPU time. 2024-02-10T07:45:31.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:31.251 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-10T07:45:31.251 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-02-10T07:45:31.251 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.z 2024-02-10T07:45:31.597 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:45:31 smithi081 systemd[1]: Stopping Ceph mgr.z for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:31.936 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:45:31 smithi081 podman[43651]: 2024-02-10 07:45:31.659067735 +0000 UTC m=+0.217403716 container died c138fc9611131149c144735f6742638539d5fbc720bdaf9d8599312e275e6551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.2, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, RELEASE=main-290212a, org.label-schema.license=GPLv2) 2024-02-10T07:45:32.310 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:45:32 smithi081 podman[43651]: 2024-02-10 07:45:32.009775591 +0000 UTC m=+0.568111568 container cleanup c138fc9611131149c144735f6742638539d5fbc720bdaf9d8599312e275e6551 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z, org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=main-290212a) 2024-02-10T07:45:32.310 INFO:journalctl@ceph.mgr.z.smithi081.stdout:Feb 10 07:45:32 smithi081 bash[43651]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-mgr-z 2024-02-10T07:45:32.444 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@mgr.z.service' 2024-02-10T07:45:32.480 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:32.480 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-02-10T07:45:32.480 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-10T07:45:32.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.0 2024-02-10T07:45:32.825 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:32 smithi022 systemd[1]: Stopping Ceph osd.0 for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:33.194 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0[43841]: 2024-02-10T07:45:32.824+0000 7f1049793700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:33.194 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0[43841]: 2024-02-10T07:45:32.824+0000 7f1049793700 -1 osd.0 31 *** Got signal Terminated *** 2024-02-10T07:45:33.194 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:32 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0[43841]: 2024-02-10T07:45:32.824+0000 7f1049793700 -1 osd.0 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:45:38.406 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:38 smithi022 podman[60436]: 2024-02-10 07:45:38.135344487 +0000 UTC m=+5.466386687 container died 690f17a4facb33ca9026e43561b5357b340a31a47445500fe877148341fbed7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS, RELEASE=main-290212a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-02-10T07:45:38.694 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:38 smithi022 podman[60436]: 2024-02-10 07:45:38.579980315 +0000 UTC m=+5.911022514 container cleanup 690f17a4facb33ca9026e43561b5357b340a31a47445500fe877148341fbed7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0, org.label-schema.license=GPLv2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240131, RELEASE=main-290212a) 2024-02-10T07:45:38.694 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:38 smithi022 bash[60436]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0 2024-02-10T07:45:39.194 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:38 smithi022 podman[60470]: 2024-02-10 07:45:38.814752896 +0000 UTC m=+0.674930322 container remove 690f17a4facb33ca9026e43561b5357b340a31a47445500fe877148341fbed7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.2, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131) 2024-02-10T07:45:39.559 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.159690429 +0000 UTC m=+0.018130741 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:45:39.559 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.265409091 +0000 UTC m=+0.123849429 container create 6cd67d697d6e3fd632a2bbd41365b2b5c4cd9b6e35e1dacb7fa28615d0e9956a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-02-10T07:45:39.559 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.467814069 +0000 UTC m=+0.326254379 container init 6cd67d697d6e3fd632a2bbd41365b2b5c4cd9b6e35e1dacb7fa28615d0e9956a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0-deactivate, org.label-schema.build-date=20240131, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.vendor=CentOS) 2024-02-10T07:45:39.559 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.473074227 +0000 UTC m=+0.331514522 container start 6cd67d697d6e3fd632a2bbd41365b2b5c4cd9b6e35e1dacb7fa28615d0e9956a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.2, RELEASE=main-290212a) 2024-02-10T07:45:39.559 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.559244401 +0000 UTC m=+0.417684708 container attach 6cd67d697d6e3fd632a2bbd41365b2b5c4cd9b6e35e1dacb7fa28615d0e9956a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2) 2024-02-10T07:45:39.944 INFO:journalctl@ceph.osd.0.smithi022.stdout:Feb 10 07:45:39 smithi022 podman[60569]: 2024-02-10 07:45:39.769066769 +0000 UTC m=+0.627507071 container died 6cd67d697d6e3fd632a2bbd41365b2b5c4cd9b6e35e1dacb7fa28615d0e9956a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131, GIT_CLEAN=True, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-02-10T07:45:40.480 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.0.service' 2024-02-10T07:45:40.527 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:40.528 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-10T07:45:40.528 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-10T07:45:40.528 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.1 2024-02-10T07:45:40.868 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:40 smithi027 systemd[1]: Stopping Ceph osd.1 for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:41.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1[34423]: 2024-02-10T07:45:40.867+0000 7fd1f2ab4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:41.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1[34423]: 2024-02-10T07:45:40.867+0000 7fd1f2ab4700 -1 osd.1 31 *** Got signal Terminated *** 2024-02-10T07:45:41.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:40 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1[34423]: 2024-02-10T07:45:40.867+0000 7fd1f2ab4700 -1 osd.1 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:45:46.675 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:46 smithi027 podman[43625]: 2024-02-10 07:45:46.39473339 +0000 UTC m=+5.653708081 container died dcb2deb494ef3b2bd075f432ddb5ea9e7aaad2290381615e64310a3dbfd15584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=main-290212a) 2024-02-10T07:45:47.015 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:46 smithi027 podman[43625]: 2024-02-10 07:45:46.756347731 +0000 UTC m=+6.015322471 container cleanup dcb2deb494ef3b2bd075f432ddb5ea9e7aaad2290381615e64310a3dbfd15584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1, io.buildah.version=1.33.2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-02-10T07:45:47.015 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:46 smithi027 bash[43625]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1 2024-02-10T07:45:47.374 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43648]: 2024-02-10 07:45:47.015396745 +0000 UTC m=+0.617096776 container remove dcb2deb494ef3b2bd075f432ddb5ea9e7aaad2290381615e64310a3dbfd15584 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-02-10T07:45:47.676 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.282835515 +0000 UTC m=+0.019286119 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:45:47.676 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.382933614 +0000 UTC m=+0.119384274 container create 67a0c90674038c0d1794f65392e573cb6088a93f5f38a85c916272154e8c9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240131, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-02-10T07:45:47.676 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.593526149 +0000 UTC m=+0.329976750 container init 67a0c90674038c0d1794f65392e573cb6088a93f5f38a85c916272154e8c9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=main-290212a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:45:47.676 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.599120873 +0000 UTC m=+0.335571474 container start 67a0c90674038c0d1794f65392e573cb6088a93f5f38a85c916272154e8c9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, RELEASE=main-290212a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240131) 2024-02-10T07:45:47.676 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.676501973 +0000 UTC m=+0.412952583 container attach 67a0c90674038c0d1794f65392e573cb6088a93f5f38a85c916272154e8c9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-290212a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_BRANCH=HEAD) 2024-02-10T07:45:48.124 INFO:journalctl@ceph.osd.1.smithi027.stdout:Feb 10 07:45:47 smithi027 podman[43745]: 2024-02-10 07:45:47.908376595 +0000 UTC m=+0.644827208 container died 67a0c90674038c0d1794f65392e573cb6088a93f5f38a85c916272154e8c9987 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=main-290212a) 2024-02-10T07:45:48.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.1.service' 2024-02-10T07:45:48.550 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:48.550 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-10T07:45:48.550 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-10T07:45:48.550 DEBUG:teuthology.orchestra.run.smithi081:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.2 2024-02-10T07:45:48.847 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:48 smithi081 systemd[1]: Stopping Ceph osd.2 for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:49.347 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:49 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2[35491]: 2024-02-10T07:45:49.056+0000 7f17371b7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:49.348 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:49 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2[35491]: 2024-02-10T07:45:49.056+0000 7f17371b7700 -1 osd.2 31 *** Got signal Terminated *** 2024-02-10T07:45:49.348 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:49 smithi081 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2[35491]: 2024-02-10T07:45:49.056+0000 7f17371b7700 -1 osd.2 31 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-10T07:45:54.647 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:54 smithi081 podman[43798]: 2024-02-10 07:45:54.391969446 +0000 UTC m=+5.644169190 container died 7ef97597a88ad2266e18e06deb208af69efdfd6b106021e5d4183ddb6430db0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:45:54.922 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:54 smithi081 podman[43798]: 2024-02-10 07:45:54.73721533 +0000 UTC m=+5.989415073 container cleanup 7ef97597a88ad2266e18e06deb208af69efdfd6b106021e5d4183ddb6430db0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2, RELEASE=main-290212a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=) 2024-02-10T07:45:54.922 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:54 smithi081 bash[43798]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2 2024-02-10T07:45:55.338 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:54 smithi081 podman[43822]: 2024-02-10 07:45:54.921738657 +0000 UTC m=+0.524372616 container remove 7ef97597a88ad2266e18e06deb208af69efdfd6b106021e5d4183ddb6430db0a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2, org.label-schema.license=GPLv2, RELEASE=main-290212a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, org.label-schema.vendor=CentOS) 2024-02-10T07:45:55.597 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.339195292 +0000 UTC m=+0.105902975 container create 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.license=GPLv2, RELEASE=main-290212a, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-02-10T07:45:55.598 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.250397763 +0000 UTC m=+0.017105387 image pull c328ab9dcfc7bf907ed3835578853a1e5a54ac5fa4d658ed833b2bc74e1373a6 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58 2024-02-10T07:45:55.598 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.534963806 +0000 UTC m=+0.301671425 container init 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.build-date=20240131, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.2, GIT_BRANCH=HEAD) 2024-02-10T07:45:55.598 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.540796968 +0000 UTC m=+0.307504597 container start 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_CLEAN=True, org.label-schema.build-date=20240131, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2) 2024-02-10T07:45:56.097 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.616228058 +0000 UTC m=+0.382935676 container attach 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, RELEASE=main-290212a, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.2, ceph=True, org.label-schema.build-date=20240131, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-02-10T07:45:56.097 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:55 smithi081 podman[43921]: 2024-02-10 07:45:55.834301327 +0000 UTC m=+0.601008949 container died 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240131, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=main-290212a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.2, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True) 2024-02-10T07:45:56.369 INFO:journalctl@ceph.osd.2.smithi081.stdout:Feb 10 07:45:56 smithi081 podman[43966]: 2024-02-10 07:45:56.352373814 +0000 UTC m=+0.513027306 container remove 0f68d65f0f350ec8172d640153ed0250d74c0d2669fe06d3ca5b290860e5f83a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240131, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, RELEASE=main-290212a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-02-10T07:45:56.375 DEBUG:teuthology.orchestra.run.smithi081:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@osd.2.service' 2024-02-10T07:45:56.415 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:45:56.415 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-10T07:45:56.416 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-10T07:45:56.416 DEBUG:teuthology.orchestra.run.smithi022:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@rgw.foo.a 2024-02-10T07:45:56.724 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:45:56 smithi022 systemd[1]: Stopping Ceph rgw.foo.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:45:57.194 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:45:56 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a[50663]: 2024-02-10T07:45:56.723+0000 7f419deea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-02-10T07:45:57.194 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:45:56 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a[50663]: 2024-02-10T07:45:56.723+0000 7f41ac75bac0 -1 shutting down 2024-02-10T07:46:07.104 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:46:06 smithi022 bash[60701]: time="2024-02-10T07:46:06Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-02-10T07:46:07.104 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:46:06 smithi022 podman[60701]: 2024-02-10 07:46:06.749950281 +0000 UTC m=+10.139089260 container died ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240131, RELEASE=main-290212a, io.buildah.version=1.33.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275) 2024-02-10T07:46:07.444 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:46:07 smithi022 podman[60701]: 2024-02-10 07:46:07.236998926 +0000 UTC m=+10.626137905 container cleanup ee3e3e840ba54d12c58bae2ea0910fc57e9be1377af0843d593ba8d994a96fc5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:caf5e50b0be996bed0eb0d30a5482e8034654966f88a3565d36aeb446466fb58, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a, org.label-schema.build-date=20240131, ceph=True, RELEASE=main-290212a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=14dafef590928bb89728de91784b87b8f9bbf275, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-02-10T07:46:07.444 INFO:journalctl@ceph.rgw.foo.a.smithi022.stdout:Feb 10 07:46:07 smithi022 bash[60701]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-rgw-foo-a 2024-02-10T07:46:07.660 DEBUG:teuthology.orchestra.run.smithi022:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@rgw.foo.a.service' 2024-02-10T07:46:07.696 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:46:07.696 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-10T07:46:07.696 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-10T07:46:07.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@prometheus.a 2024-02-10T07:46:08.008 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:07 smithi027 systemd[1]: Stopping Ceph prometheus.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:08.289 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:46:08.289 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-02-10T07:46:08.289 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-02-10T07:46:08.289 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-10T07:46:08.289 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.008Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.009Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.009Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a[41033]: ts=2024-02-10T07:46:08.009Z caller=main.go:1147 level=info msg="See you next time!" 2024-02-10T07:46:08.290 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 podman[43873]: 2024-02-10 07:46:08.018864091 +0000 UTC m=+0.124992854 container died fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:08.624 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 podman[43873]: 2024-02-10 07:46:08.396735611 +0000 UTC m=+0.502864376 container cleanup fe7fa7590d3f683c4094abc074121238a54ec3603dabde1ec199009a55811e7d (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:08.624 INFO:journalctl@ceph.prometheus.a.smithi027.stdout:Feb 10 07:46:08 smithi027 bash[43873]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-prometheus-a 2024-02-10T07:46:08.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo pkill -f 'journalctl -f -n 0 -u ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@prometheus.a.service' 2024-02-10T07:46:08.802 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-10T07:46:08.803 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-10T07:46:08.803 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T07:46:08.997 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:10.178 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:09 smithi022 systemd[1]: Stopping Ceph alertmanager.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:10.444 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a[57839]: ts=2024-02-10T07:46:10.177Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-02-10T07:46:10.444 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 podman[61055]: 2024-02-10 07:46:10.198206774 +0000 UTC m=+0.129185817 container died 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:10.800 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 podman[61055]: 2024-02-10 07:46:10.608397126 +0000 UTC m=+0.539376168 container cleanup 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:10.801 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 bash[61055]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a 2024-02-10T07:46:11.194 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 podman[61074]: 2024-02-10 07:46:10.801128969 +0000 UTC m=+0.605915458 container remove 5f5e0eb2c56b9d12288f48a1806f966174442fbe854f52812e1e6bc17b15439b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-alertmanager-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:11.194 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:10 smithi022 podman[61074]: 2024-02-10 07:46:10.885066 +0000 UTC m=+0.689852456 volume remove 003f2083e0a7bf553983abee14c21e53d533e271b140a851c9bff9767a5c513d 2024-02-10T07:46:11.194 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:11 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-02-10T07:46:11.194 INFO:journalctl@ceph.alertmanager.a.smithi022.stdout:Feb 10 07:46:11 smithi022 systemd[1]: Stopped Ceph alertmanager.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:46:11.694 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:11 smithi022 systemd[1]: Stopping Ceph node-exporter.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:11.694 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:11 smithi022 podman[61192]: 2024-02-10 07:46:11.45962051 +0000 UTC m=+0.094020891 container died d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:12.113 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:11 smithi022 podman[61192]: 2024-02-10 07:46:11.862403046 +0000 UTC m=+0.496803466 container cleanup d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:12.113 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:11 smithi022 bash[61192]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a 2024-02-10T07:46:12.444 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:12 smithi022 podman[61211]: 2024-02-10 07:46:12.113595815 +0000 UTC m=+0.659145731 container remove d728710ab643ef582604014568d4cef013d945cd84b8f89c0052a433d0930c6c (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-a, maintainer=The Prometheus Authors ) 2024-02-10T07:46:12.444 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:12 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:46:12.444 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:12 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-10T07:46:12.444 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:12 smithi022 systemd[1]: Stopped Ceph node-exporter.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:46:12.444 INFO:journalctl@ceph.node-exporter.a.smithi022.stdout:Feb 10 07:46:12 smithi022 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.a.service: Consumed 1.779s CPU time. 2024-02-10T07:46:12.947 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T07:46:13.150 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:14.314 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 systemd[1]: Stopping Ceph grafana.a for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:14.624 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 podman[44223]: 2024-02-10 07:46:14.314560252 +0000 UTC m=+0.109853087 container died 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, version=8.5, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.k8s.display-name=Red Hat Universal Base Image 8, architecture=x86_64, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, maintainer=Paul Cuzner , name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, build-date=2022-04-29T11:48:11.055897, com.redhat.component=ubi8-container, io.openshift.expose-services=, vcs-type=git, distribution-scope=public, io.buildah.version=1.32.0, io.openshift.tags=base rhel8, release=239.1651231664, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, description=Ceph Grafana Container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-02-10T07:46:15.034 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 podman[44223]: 2024-02-10 07:46:14.624957072 +0000 UTC m=+0.420249908 container cleanup 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, io.buildah.version=1.32.0, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, version=8.5, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, io.openshift.tags=base rhel8, release=239.1651231664, vendor=Red Hat, Inc., com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, maintainer=Paul Cuzner , com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, description=Ceph Grafana Container, architecture=x86_64, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-type=git, io.openshift.expose-services=, name=ubi8, io.k8s.display-name=Red Hat Universal Base Image 8) 2024-02-10T07:46:15.034 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 bash[44223]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a 2024-02-10T07:46:15.034 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 podman[44242]: 2024-02-10 07:46:14.809630671 +0000 UTC m=+0.493383493 container remove 3e06af8ae3f446ad59a43db616267cdfd451f89edda85e68469d7cff97497bcf (image=quay.io/ceph/ceph-grafana:9.4.12, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-grafana-a, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., vendor=Red Hat, Inc., io.openshift.expose-services=, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, release=239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, summary=Grafana Container configured for Ceph mgr/dashboard integration, architecture=x86_64, maintainer=Paul Cuzner , version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, build-date=2022-04-29T11:48:11.055897, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.buildah.version=1.32.0, name=ubi8, com.redhat.component=ubi8-container, description=Ceph Grafana Container, distribution-scope=public, vcs-type=git) 2024-02-10T07:46:15.034 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:14 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:46:15.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:15 smithi027 systemd[1]: Stopping Ceph node-exporter.b for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:15.374 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:15 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-02-10T07:46:15.374 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:15 smithi027 systemd[1]: Stopped Ceph grafana.a for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:46:15.374 INFO:journalctl@ceph.grafana.a.smithi027.stdout:Feb 10 07:46:15 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@grafana.a.service: Consumed 2.830s CPU time. 2024-02-10T07:46:15.779 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:15 smithi027 podman[44362]: 2024-02-10 07:46:15.509432584 +0000 UTC m=+0.117022972 container died ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:46:16.112 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:15 smithi027 podman[44362]: 2024-02-10 07:46:15.878873544 +0000 UTC m=+0.486463934 container cleanup ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:46:16.112 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:15 smithi027 bash[44362]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b 2024-02-10T07:46:16.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:16 smithi027 podman[44381]: 2024-02-10 07:46:16.113040525 +0000 UTC m=+0.608415214 container remove ad5cc6ff1b2c3381386f4aaecf3683f16fb1fd7a91d3e08478c65209d5068b36 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-b, maintainer=The Prometheus Authors ) 2024-02-10T07:46:16.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:16 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:46:16.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:16 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-10T07:46:16.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:16 smithi027 systemd[1]: Stopped Ceph node-exporter.b for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:46:16.374 INFO:journalctl@ceph.node-exporter.b.smithi027.stdout:Feb 10 07:46:16 smithi027 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.b.service: Consumed 1.772s CPU time. 2024-02-10T07:46:16.901 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force --keep-logs 2024-02-10T07:46:17.090 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:18.097 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:17 smithi081 systemd[1]: Stopping Ceph node-exporter.c for 80815fa4-c7e7-11ee-95b8-87774f69a715... 2024-02-10T07:46:18.097 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:17 smithi081 podman[44216]: 2024-02-10 07:46:17.958975426 +0000 UTC m=+0.099764713 container died 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:46:18.571 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 podman[44216]: 2024-02-10 07:46:18.345287583 +0000 UTC m=+0.486076867 container cleanup 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:46:18.571 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 bash[44216]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c 2024-02-10T07:46:18.847 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 podman[44235]: 2024-02-10 07:46:18.571341599 +0000 UTC m=+0.617349311 container remove 61c93977f97fdd589bb05cb652308bc9ed0a02ed61886f7b8a52455894f2ec27 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-80815fa4-c7e7-11ee-95b8-87774f69a715-node-exporter-c, maintainer=The Prometheus Authors ) 2024-02-10T07:46:18.848 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.c.service: Main process exited, code=exited, status=143/n/a 2024-02-10T07:46:18.848 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.c.service: Failed with result 'exit-code'. 2024-02-10T07:46:18.848 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 systemd[1]: Stopped Ceph node-exporter.c for 80815fa4-c7e7-11ee-95b8-87774f69a715. 2024-02-10T07:46:18.848 INFO:journalctl@ceph.node-exporter.c.smithi081.stdout:Feb 10 07:46:18 smithi081 systemd[1]: ceph-80815fa4-c7e7-11ee-95b8-87774f69a715@node-exporter.c.service: Consumed 1.702s CPU time. 2024-02-10T07:46:19.344 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:46:19.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:46:19.419 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-10T07:46:19.450 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-10T07:46:19.452 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi022/crash 2024-02-10T07:46:19.453 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash -- . 2024-02-10T07:46:19.497 INFO:teuthology.orchestra.run.smithi022.stderr:tar: /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T07:46:19.497 INFO:teuthology.orchestra.run.smithi022.stderr:tar: Error is not recoverable: exiting now 2024-02-10T07:46:19.499 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi027/crash 2024-02-10T07:46:19.500 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash -- . 2024-02-10T07:46:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:tar: /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T07:46:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:tar: Error is not recoverable: exiting now 2024-02-10T07:46:19.552 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi081/crash 2024-02-10T07:46:19.553 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash -- . 2024-02-10T07:46:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:tar: /var/lib/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/crash: Cannot open: No such file or directory 2024-02-10T07:46:19.616 INFO:teuthology.orchestra.run.smithi081.stderr:tar: Error is not recoverable: exiting now 2024-02-10T07:46:19.618 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-10T07:46:19.618 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T07:46:19.657 INFO:teuthology.orchestra.run.smithi022.stdout:2024-02-10T07:42:24.982688+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T07:46:19.658 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-10T07:46:19.658 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[SEC\]' /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T07:46:19.730 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[ERR\]' /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T07:46:19.801 DEBUG:teuthology.orchestra.run.smithi022:> sudo egrep '\[WRN\]' /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-02-10T07:46:19.871 INFO:teuthology.orchestra.run.smithi022.stdout:2024-02-10T07:42:24.982688+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-02-10T07:46:19.871 INFO:tasks.cephadm:Compressing logs... 2024-02-10T07:46:19.872 DEBUG:teuthology.orchestra.run.smithi022:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T07:46:19.914 DEBUG:teuthology.orchestra.run.smithi027:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T07:46:19.916 DEBUG:teuthology.orchestra.run.smithi081:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-10T07:46:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T07:46:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:: No such file or directory 2024-02-10T07:46:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.a.log 2024-02-10T07:46:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T07:46:19.940 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-10T07:46:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.y.log 2024-02-10T07:46:19.942 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.0.log 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr: 57.7% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr: 89.4% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr: 90.6% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log: 82.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T07:46:19.943 INFO:teuthology.orchestra.run.smithi022.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log: 85.1% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:: No such file or directory 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.z.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log: /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr: 90.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.2.log 2024-02-10T07:46:19.946 INFO:teuthology.orchestra.run.smithi081.stderr: 81.0% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T07:46:19.947 INFO:teuthology.orchestra.run.smithi081.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T07:46:19.947 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log: /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.c.log 2024-02-10T07:46:19.947 INFO:teuthology.orchestra.run.smithi081.stderr: 89.6% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T07:46:19.947 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log: 84.7% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T07:46:19.947 INFO:teuthology.orchestra.run.smithi022.stderr: 90.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T07:46:19.948 INFO:teuthology.orchestra.run.smithi081.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.c.log: 90.1% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.z.log.gz 2024-02-10T07:46:19.948 INFO:teuthology.orchestra.run.smithi027.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-10T07:46:19.949 INFO:teuthology.orchestra.run.smithi027.stderr:: No such file or directory 2024-02-10T07:46:19.949 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log 2024-02-10T07:46:19.949 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.1.log 2024-02-10T07:46:19.950 INFO:teuthology.orchestra.run.smithi081.stderr: 90.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T07:46:19.951 INFO:teuthology.orchestra.run.smithi027.stderr:gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.x.log 2024-02-10T07:46:19.951 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log: /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log 2024-02-10T07:46:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log 2024-02-10T07:46:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log 2024-02-10T07:46:19.953 INFO:teuthology.orchestra.run.smithi027.stderr: 89.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.audit.log.gz 2024-02-10T07:46:19.953 INFO:teuthology.orchestra.run.smithi027.stderr: 81.1% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.cephadm.log.gz 2024-02-10T07:46:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.b.log 2024-02-10T07:46:19.953 INFO:teuthology.orchestra.run.smithi027.stderr: 81.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-10T07:46:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log: 84.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph.log.gz 2024-02-10T07:46:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:/var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.b.log: 90.1% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.x.log.gz 2024-02-10T07:46:19.955 INFO:teuthology.orchestra.run.smithi027.stderr: 90.5% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-volume.log.gz 2024-02-10T07:46:20.006 INFO:teuthology.orchestra.run.smithi081.stderr: 92.2% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.c.log.gz 2024-02-10T07:46:20.007 INFO:teuthology.orchestra.run.smithi027.stderr: 92.4% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.b.log.gz 2024-02-10T07:46:20.007 INFO:teuthology.orchestra.run.smithi022.stderr: 88.9% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mgr.y.log.gz 2024-02-10T07:46:20.123 INFO:teuthology.orchestra.run.smithi022.stderr: 91.4% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-mon.a.log.gz 2024-02-10T07:46:20.197 INFO:teuthology.orchestra.run.smithi081.stderr: 94.3% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.2.log.gz 2024-02-10T07:46:20.199 INFO:teuthology.orchestra.run.smithi081.stderr: 2024-02-10T07:46:20.199 INFO:teuthology.orchestra.run.smithi081.stderr:real 0m0.270s 2024-02-10T07:46:20.199 INFO:teuthology.orchestra.run.smithi081.stderr:user 0m0.325s 2024-02-10T07:46:20.200 INFO:teuthology.orchestra.run.smithi081.stderr:sys 0m0.034s 2024-02-10T07:46:20.204 INFO:teuthology.orchestra.run.smithi022.stderr: 94.3% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.0.log.gz 2024-02-10T07:46:20.205 INFO:teuthology.orchestra.run.smithi022.stderr: 2024-02-10T07:46:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:real 0m0.279s 2024-02-10T07:46:20.205 INFO:teuthology.orchestra.run.smithi022.stderr:user 0m0.508s 2024-02-10T07:46:20.206 INFO:teuthology.orchestra.run.smithi022.stderr:sys 0m0.046s 2024-02-10T07:46:20.268 INFO:teuthology.orchestra.run.smithi027.stderr: 94.3% -- replaced with /var/log/ceph/80815fa4-c7e7-11ee-95b8-87774f69a715/ceph-osd.1.log.gz 2024-02-10T07:46:20.270 INFO:teuthology.orchestra.run.smithi027.stderr: 2024-02-10T07:46:20.270 INFO:teuthology.orchestra.run.smithi027.stderr:real 0m0.339s 2024-02-10T07:46:20.270 INFO:teuthology.orchestra.run.smithi027.stderr:user 0m0.397s 2024-02-10T07:46:20.271 INFO:teuthology.orchestra.run.smithi027.stderr:sys 0m0.033s 2024-02-10T07:46:20.271 INFO:tasks.cephadm:Archiving logs... 2024-02-10T07:46:20.271 DEBUG:teuthology.misc:Transferring archived files from smithi022:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi022/log 2024-02-10T07:46:20.272 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T07:46:20.386 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi027/log 2024-02-10T07:46:20.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T07:46:20.479 DEBUG:teuthology.misc:Transferring archived files from smithi081:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi081/log 2024-02-10T07:46:20.479 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-10T07:46:20.558 INFO:tasks.cephadm:Removing cluster... 2024-02-10T07:46:20.559 DEBUG:teuthology.orchestra.run.smithi022:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force 2024-02-10T07:46:20.741 INFO:teuthology.orchestra.run.smithi022.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:20.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force 2024-02-10T07:46:21.042 INFO:teuthology.orchestra.run.smithi027.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:21.162 DEBUG:teuthology.orchestra.run.smithi081:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 80815fa4-c7e7-11ee-95b8-87774f69a715 --force 2024-02-10T07:46:21.348 INFO:teuthology.orchestra.run.smithi081.stdout:Deleting cluster with fsid: 80815fa4-c7e7-11ee-95b8-87774f69a715 2024-02-10T07:46:21.459 INFO:tasks.cephadm:Removing cephadm ... 2024-02-10T07:46:21.459 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T07:46:21.476 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T07:46:21.491 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf /home/ubuntu/cephtest/cephadm 2024-02-10T07:46:21.505 INFO:tasks.cephadm:Teardown complete 2024-02-10T07:46:21.506 DEBUG:teuthology.run_tasks:Unwinding manager nvme_loop 2024-02-10T07:46:21.515 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi022:/dev/vg_nvme/lv_1... 2024-02-10T07:46:21.515 DEBUG:teuthology.orchestra.run.smithi022:> sudo nvme disconnect -n lv_1 2024-02-10T07:46:21.662 INFO:teuthology.orchestra.run.smithi022.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T07:46:21.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:21.664 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi022:/dev/vg_nvme/lv_2... 2024-02-10T07:46:21.664 DEBUG:teuthology.orchestra.run.smithi022:> sudo nvme disconnect -n lv_2 2024-02-10T07:46:21.829 INFO:teuthology.orchestra.run.smithi022.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T07:46:21.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:21.831 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi022:/dev/vg_nvme/lv_3... 2024-02-10T07:46:21.831 DEBUG:teuthology.orchestra.run.smithi022:> sudo nvme disconnect -n lv_3 2024-02-10T07:46:21.992 INFO:teuthology.orchestra.run.smithi022.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T07:46:21.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:21.994 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi022:/dev/vg_nvme/lv_4... 2024-02-10T07:46:21.994 DEBUG:teuthology.orchestra.run.smithi022:> sudo nvme disconnect -n lv_4 2024-02-10T07:46:22.131 INFO:teuthology.orchestra.run.smithi022.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T07:46:22.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:22.133 DEBUG:teuthology.orchestra.run.smithi022:> set -ex 2024-02-10T07:46:22.133 DEBUG:teuthology.orchestra.run.smithi022:> sudo dd of=/scratch_devs 2024-02-10T07:46:22.159 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi027:/dev/vg_nvme/lv_1... 2024-02-10T07:46:22.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo nvme disconnect -n lv_1 2024-02-10T07:46:22.302 INFO:teuthology.orchestra.run.smithi027.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T07:46:22.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:22.304 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi027:/dev/vg_nvme/lv_2... 2024-02-10T07:46:22.304 DEBUG:teuthology.orchestra.run.smithi027:> sudo nvme disconnect -n lv_2 2024-02-10T07:46:22.470 INFO:teuthology.orchestra.run.smithi027.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T07:46:22.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:22.472 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi027:/dev/vg_nvme/lv_3... 2024-02-10T07:46:22.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo nvme disconnect -n lv_3 2024-02-10T07:46:22.641 INFO:teuthology.orchestra.run.smithi027.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T07:46:22.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:22.643 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi027:/dev/vg_nvme/lv_4... 2024-02-10T07:46:22.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo nvme disconnect -n lv_4 2024-02-10T07:46:22.827 INFO:teuthology.orchestra.run.smithi027.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T07:46:22.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:22.829 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2024-02-10T07:46:22.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/scratch_devs 2024-02-10T07:46:22.860 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi081:/dev/vg_nvme/lv_1... 2024-02-10T07:46:22.860 DEBUG:teuthology.orchestra.run.smithi081:> sudo nvme disconnect -n lv_1 2024-02-10T07:46:23.001 INFO:teuthology.orchestra.run.smithi081.stdout:NQN:lv_1 disconnected 1 controller(s) 2024-02-10T07:46:23.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.003 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi081:/dev/vg_nvme/lv_2... 2024-02-10T07:46:23.003 DEBUG:teuthology.orchestra.run.smithi081:> sudo nvme disconnect -n lv_2 2024-02-10T07:46:23.145 INFO:teuthology.orchestra.run.smithi081.stdout:NQN:lv_2 disconnected 1 controller(s) 2024-02-10T07:46:23.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.147 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi081:/dev/vg_nvme/lv_3... 2024-02-10T07:46:23.147 DEBUG:teuthology.orchestra.run.smithi081:> sudo nvme disconnect -n lv_3 2024-02-10T07:46:23.299 INFO:teuthology.orchestra.run.smithi081.stdout:NQN:lv_3 disconnected 1 controller(s) 2024-02-10T07:46:23.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.301 INFO:tasks.nvme_loop:Disconnecting nvme_loop smithi081:/dev/vg_nvme/lv_4... 2024-02-10T07:46:23.301 DEBUG:teuthology.orchestra.run.smithi081:> sudo nvme disconnect -n lv_4 2024-02-10T07:46:23.447 INFO:teuthology.orchestra.run.smithi081.stdout:NQN:lv_4 disconnected 1 controller(s) 2024-02-10T07:46:23.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.449 DEBUG:teuthology.orchestra.run.smithi081:> set -ex 2024-02-10T07:46:23.449 DEBUG:teuthology.orchestra.run.smithi081:> sudo dd of=/scratch_devs 2024-02-10T07:46:23.478 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-10T07:46:23.487 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-10T07:46:23.487 DEBUG:teuthology.orchestra.run.smithi022:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:46:23.490 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:46:23.492 DEBUG:teuthology.orchestra.run.smithi081:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-10T07:46:23.502 INFO:teuthology.orchestra.run.smithi022.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:46:23.504 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:46:23.507 INFO:teuthology.orchestra.run.smithi022.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:46:23.507 INFO:teuthology.orchestra.run.smithi022.stdout:=============================================================================== 2024-02-10T07:46:23.507 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 37 +660us[ +665us] +/- 88ms 2024-02-10T07:46:23.507 INFO:teuthology.orchestra.run.smithi022.stdout:^* hv02.front.sepia.ceph.com 3 6 377 36 -317us[ -311us] +/- 56ms 2024-02-10T07:46:23.508 INFO:teuthology.orchestra.run.smithi022.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -85us[ -79us] +/- 62ms 2024-02-10T07:46:23.508 INFO:teuthology.orchestra.run.smithi022.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:46:23.508 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:46:23.509 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2024-02-10T07:46:23.509 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +669us[ +669us] +/- 88ms 2024-02-10T07:46:23.509 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv02.front.sepia.ceph.com 3 6 377 37 -314us[ -310us] +/- 56ms 2024-02-10T07:46:23.509 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 37 -83us[ -83us] +/- 62ms 2024-02-10T07:46:23.509 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:46:23.534 INFO:teuthology.orchestra.run.smithi081.stderr:bash: line 1: ntpq: command not found 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:=============================================================================== 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 36 +661us[ +672us] +/- 88ms 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -322us[ -312us] +/- 56ms 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 36 -90us[ -79us] +/- 62ms 2024-02-10T07:46:23.538 INFO:teuthology.orchestra.run.smithi081.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-02-10T07:46:23.539 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-10T07:46:23.547 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-10T07:46:23.548 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-10T07:46:23.555 DEBUG:teuthology.orchestra.run.smithi022:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T07:46:23.631 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T07:46:23.706 DEBUG:teuthology.orchestra.run.smithi081:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-02-10T07:46:23.779 DEBUG:teuthology.orchestra.run.smithi022:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:46:23.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:46:23.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.844 DEBUG:teuthology.orchestra.run.smithi081:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-02-10T07:46:23.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:23.876 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-10T07:46:23.886 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-10T07:46:23.895 INFO:teuthology.task.internal:Duration was 691.228186 seconds 2024-02-10T07:46:23.896 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-10T07:46:23.904 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-10T07:46:23.905 DEBUG:teuthology.orchestra.run.smithi022:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T07:46:23.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T07:46:23.909 DEBUG:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-10T07:46:23.949 INFO:teuthology.orchestra.run.smithi022.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:46:23.957 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:46:23.960 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-02-10T07:46:24.230 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-10T07:46:24.231 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi022.front.sepia.ceph.com 2024-02-10T07:46:24.231 DEBUG:teuthology.orchestra.run.smithi022:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T07:46:24.251 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2024-02-10T07:46:24.251 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T07:46:24.279 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi081.front.sepia.ceph.com 2024-02-10T07:46:24.280 DEBUG:teuthology.orchestra.run.smithi081:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-10T07:46:24.305 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-10T07:46:24.305 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T07:46:24.307 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T07:46:24.322 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-10T07:46:24.374 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-10T07:46:24.374 DEBUG:teuthology.orchestra.run.smithi022:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T07:46:24.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T07:46:24.394 DEBUG:teuthology.orchestra.run.smithi081:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-10T07:46:24.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-10T07:46:24.579 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-10T07:46:24.579 DEBUG:teuthology.orchestra.run.smithi022:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T07:46:24.606 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T07:46:24.637 DEBUG:teuthology.orchestra.run.smithi081:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-10T07:46:24.664 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-10T07:46:24.674 DEBUG:teuthology.orchestra.run.smithi022:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.677 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.679 DEBUG:teuthology.orchestra.run.smithi081:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.700 INFO:teuthology.orchestra.run.smithi022.stdout:kernel.core_pattern = core 2024-02-10T07:46:24.706 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2024-02-10T07:46:24.766 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = core 2024-02-10T07:46:24.766 DEBUG:teuthology.orchestra.run.smithi022:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:24.820 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:24.835 DEBUG:teuthology.orchestra.run.smithi081:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-10T07:46:24.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-10T07:46:24.850 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-10T07:46:24.861 INFO:teuthology.task.internal:Transferring archived files... 2024-02-10T07:46:24.863 DEBUG:teuthology.misc:Transferring archived files from smithi022:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi022 2024-02-10T07:46:24.863 DEBUG:teuthology.orchestra.run.smithi022:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T07:46:24.902 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi027 2024-02-10T07:46:24.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T07:46:24.946 DEBUG:teuthology.misc:Transferring archived files from smithi081:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222/remote/smithi081 2024-02-10T07:46:24.946 DEBUG:teuthology.orchestra.run.smithi081:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-10T07:46:24.983 INFO:teuthology.task.internal:Removing archive directory... 2024-02-10T07:46:24.983 DEBUG:teuthology.orchestra.run.smithi022:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T07:46:24.985 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T07:46:24.987 DEBUG:teuthology.orchestra.run.smithi081:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-10T07:46:25.032 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-10T07:46:25.044 INFO:teuthology.task.internal:Not uploading archives. 2024-02-10T07:46:25.044 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-10T07:46:25.054 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-10T07:46:25.054 DEBUG:teuthology.orchestra.run.smithi022:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T07:46:25.056 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T07:46:25.058 DEBUG:teuthology.orchestra.run.smithi081:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-10T07:46:25.068 INFO:teuthology.orchestra.run.smithi022.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 07:46 /home/ubuntu/cephtest 2024-02-10T07:46:25.071 INFO:teuthology.orchestra.run.smithi027.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 07:46 /home/ubuntu/cephtest 2024-02-10T07:46:25.086 INFO:teuthology.orchestra.run.smithi081.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 10 07:46 /home/ubuntu/cephtest 2024-02-10T07:46:25.088 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-10T07:46:25.098 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-10T07:46:25.119 INFO:teuthology.nuke:Checking targets against current locks 2024-02-10T07:46:25.147 DEBUG:teuthology.nuke:shortname: smithi022 2024-02-10T07:46:25.148 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:46:25.173 DEBUG:teuthology.nuke:shortname: smithi027 2024-02-10T07:46:25.173 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:46:25.178 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi022.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.163470', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEEyxbFM6rlK+NYUbKVSETSKWLrwjA3qkXVbObPyHOnouwTftuSdmzbDWgXVZPH+MYXYvFgyKLldt5AiJQdPYZk='} 2024-02-10T07:46:25.193 DEBUG:teuthology.nuke:shortname: smithi081 2024-02-10T07:46:25.194 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-10T07:46:25.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.162075', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0r0xdU/X2dzcRgIL95qYCP2Kz7TjiODmLwlI2lMgH1G0RhiWs2Yb8pVVUQamwDZNID6qUTpMjEkkv/WxC7uIE='} 2024-02-10T07:46:25.212 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi081.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-09_19:44:16-orch-main-distro-default-smithi/7555222', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-10 07:23:59.160175', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMSdWICprRMhyy/1rUMIH2J1gYde9QewfmaduvVjk6RJn+kjD39f9CsuesEegECZlg9rRBfeewfRrMCXimVGGnA='} 2024-02-10T07:46:25.243 INFO:teuthology.orchestra.console.smithi027:Power off 2024-02-10T07:46:25.243 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T07:46:25.259 INFO:teuthology.orchestra.console.smithi081:Power off 2024-02-10T07:46:25.259 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T07:46:25.271 INFO:teuthology.orchestra.console.smithi022:Power off 2024-02-10T07:46:25.272 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-10T07:46:25.281 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2024-02-10T07:46:25.282 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:25.290 DEBUG:teuthology.orchestra.console.smithi081:power off output: Chassis Power Control: Down/Off 2024-02-10T07:46:25.290 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:25.298 DEBUG:teuthology.orchestra.console.smithi022:power off output: Chassis Power Control: Down/Off 2024-02-10T07:46:25.299 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:25.308 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-10T07:46:25.311 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-02-10T07:46:25.324 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-02-10T07:46:29.310 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:29.319 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:29.326 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:29.422 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-10T07:46:29.429 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is on 2024-02-10T07:46:29.435 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is on 2024-02-10T07:46:33.425 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:33.433 DEBUG:teuthology.orchestra.console.smithi081:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:33.440 DEBUG:teuthology.orchestra.console.smithi022:pexpect command: ipmitool -H smithi022.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:33.536 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2024-02-10T07:46:33.542 DEBUG:teuthology.orchestra.console.smithi081:check power output: Chassis Power is off 2024-02-10T07:46:33.549 DEBUG:teuthology.orchestra.console.smithi022:check power output: Chassis Power is off 2024-02-10T07:46:33.643 INFO:teuthology.orchestra.console.smithi081:Power off completed 2024-02-10T07:46:33.650 INFO:teuthology.orchestra.console.smithi022:Power off completed 2024-02-10T07:46:37.540 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-10T07:46:37.648 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2024-02-10T07:46:37.749 INFO:teuthology.orchestra.console.smithi027:Power off completed 2024-02-10T07:46:37.853 INFO:teuthology.run:Summary data: description: orch/cephadm/smoke-small/{0-distro/centos_9.stream_runc 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} duration: 691.2281861305237 failure_reason: '"2024-02-10T07:42:24.982688+0000 mon.a (mon.0) 192 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,c" in cluster log' owner: scheduled_yuriw@teuthology success: false 2024-02-10T07:46:37.853 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-10T07:46:37.929 INFO:teuthology.run:FAIL